-- Logs begin at Sun 2022-09-25 00:10:34 UTC, end at Sun 2022-09-25 01:23:03 UTC. -- Sep 25 00:42:34.367164 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Sep 25 00:42:34.382664 dv-cinder-master-3-smb devstack@keystone.service[81488]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 25 00:42:34 2022] *** Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: nodename: dv-cinder-master-3-smb Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: machine: x86_64 Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: clock source: unix Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: pcre jit disabled Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: detected number of CPU cores: 4 Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: current working directory: / Sep 25 00:42:34.386065 dv-cinder-master-3-smb devstack@keystone.service[81488]: detected binary path: /usr/bin/uwsgi-core Sep 25 00:42:34.387051 dv-cinder-master-3-smb devstack@keystone.service[81488]: your processes number limit is 47680 Sep 25 00:42:34.387051 dv-cinder-master-3-smb devstack@keystone.service[81488]: your memory page size is 4096 bytes Sep 25 00:42:34.387051 dv-cinder-master-3-smb devstack@keystone.service[81488]: detected max file descriptor number: 2048 Sep 25 00:42:34.387051 dv-cinder-master-3-smb devstack@keystone.service[81488]: lock engine: pthread robust mutexes Sep 25 00:42:34.387051 dv-cinder-master-3-smb devstack@keystone.service[81488]: thunder lock: enabled Sep 25 00:42:34.387051 dv-cinder-master-3-smb devstack@keystone.service[81488]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 25 00:42:34.387051 dv-cinder-master-3-smb devstack@keystone.service[81488]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 25 00:42:34.429272 dv-cinder-master-3-smb devstack@keystone.service[81488]: Python main interpreter initialized at 0x55dcb714a9c0 Sep 25 00:42:34.429272 dv-cinder-master-3-smb devstack@keystone.service[81488]: python threads support enabled Sep 25 00:42:34.429272 dv-cinder-master-3-smb devstack@keystone.service[81488]: your server socket listen backlog is limited to 100 connections Sep 25 00:42:34.429272 dv-cinder-master-3-smb devstack@keystone.service[81488]: your mercy for graceful operations on workers is 90 seconds Sep 25 00:42:34.429874 dv-cinder-master-3-smb devstack@keystone.service[81488]: mapped 403077 bytes (393 KB) for 2 cores Sep 25 00:42:34.429874 dv-cinder-master-3-smb devstack@keystone.service[81488]: *** Operational MODE: preforking *** Sep 25 00:42:34.430113 dv-cinder-master-3-smb devstack@keystone.service[81488]: *** uWSGI is running in multiple interpreter mode *** Sep 25 00:42:34.430113 dv-cinder-master-3-smb devstack@keystone.service[81488]: spawned uWSGI master process (pid: 81488) Sep 25 00:42:34.430281 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Sep 25 00:42:34.433642 dv-cinder-master-3-smb devstack@keystone.service[81488]: spawned uWSGI worker 1 (pid: 81499, cores: 1) Sep 25 00:42:34.433642 dv-cinder-master-3-smb devstack@keystone.service[81488]: spawned uWSGI worker 2 (pid: 81500, cores: 1) Sep 25 00:42:34.433642 dv-cinder-master-3-smb devstack@keystone.service[81488]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 25 00:42:41.866234 dv-cinder-master-3-smb devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:42:41.866234 dv-cinder-master-3-smb devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Sep 25 00:42:41.867407 dv-cinder-master-3-smb devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:42:41.867407 dv-cinder-master-3-smb devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Sep 25 00:42:41.898400 dv-cinder-master-3-smb devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:42:41.898400 dv-cinder-master-3-smb devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Sep 25 00:42:41.899343 dv-cinder-master-3-smb devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:42:41.899343 dv-cinder-master-3-smb devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Sep 25 00:42:41.922257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 25 00:42:41.922519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 25 00:42:41.922702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] command line args: [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 25 00:42:41.923064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 25 00:42:41.923301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ================================================================================ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 25 00:42:41.923641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.923949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] config_dir = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.924192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.924482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] config_source = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.924727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] debug = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.924992 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.925701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.925701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.926044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.926459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.926859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.927178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] log_config_append = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.927495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.927810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] log_dir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.928107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] log_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.928562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.928988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.929346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.929718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.929973 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.930247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.930594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.930950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.931292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.931690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.931965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.932260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.932574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.932863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.933152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.933702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.933991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] publish_errors = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.934200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.934402 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.934603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.934897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.935177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.935982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] standard_threads = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.936254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.936570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.936785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.937002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] use_journal = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.937219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] use_json = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.937416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] use_stderr = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.937637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] use_syslog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.937999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.938275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.938588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.938840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.939123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.939411 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.939722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.939981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.940257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.940582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.940871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.941164 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.941440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.941753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.942028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.942316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.942605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.942880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.943174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.943468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.943751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.944023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.944315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.944725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.945016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.945314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.945583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.945858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.946125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.946398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.946684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.947022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.947326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.947594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.947878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.948180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.948501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.948793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.949065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.949350 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.949621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.949887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.950153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.950418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.950684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.951573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.951866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.952137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.952449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.952724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.952996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.953267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.953587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.953881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.954158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.954430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.954705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.954977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.955253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.955522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.955814 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.956086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.956359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.956684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.956988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.957261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.957543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.957820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.958182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.958495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.958774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.959144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.959444 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.959726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.960004 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.960283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.960618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.960902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.961184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.961476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.961793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.962040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.962323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.962602 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.962881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.963157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.963436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.963741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.964026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.964317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.964639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.964914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.965197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.965484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.965759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.966035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.966306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.966601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.966869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.967146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.967414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.967698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.967996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.968349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.968683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.968988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.969299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.969596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.969875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.external = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.970149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.970497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.970776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.971049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.password = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.971323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.token = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.971613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.971894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.972187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.972496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.972785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.973076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.973357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.973630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.973920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.974209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.974493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.974788 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.975070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.975353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.975672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.975965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.976194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.976544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.976875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.977164 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.977475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.977757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.978052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.978339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.978642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.978938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.979228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.979502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.979795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.980079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.980427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.980699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.980996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.981283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.981575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.981850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.982176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.982464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.982739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.983012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.983300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.983570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.983848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.984133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.984468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.984751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.985026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.985610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.985610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.986059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.986486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.987071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.987353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.987652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.987945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.988270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.988703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.988703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.989240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.989240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.989855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.989855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.990451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.990451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.991059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.991360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 25 00:42:41.991671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.991985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 25 00:42:41.992315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.992315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.992820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] command line args: [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 25 00:42:41.993056 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.993247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 25 00:42:41.993482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.993482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.993912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ================================================================================ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 25 00:42:41.994161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.994350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.994538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.994538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.994908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] config_dir = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.995097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.995276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.995506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.995686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] config_source = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.995861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.996046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] debug = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.996229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.996441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.996666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.996871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.997072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.997258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.997433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.997433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.998716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.998911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.998911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.999304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.999497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:41.999686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:41.999886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.000074 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] log_config_append = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.000272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.000478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.000665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.000848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] log_dir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.001037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.001240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] log_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.001447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.001628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.001816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.001816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.002160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.002333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.002534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.002725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.002725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.003054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.003261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.003460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.003636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.003829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.004020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.004020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.004372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.004604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.004778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.004976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.004976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.005317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.005503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.005688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.005888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.005888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.006240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.006429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.006615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.006808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.006808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.007158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.007337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.007337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.007652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.007843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.008029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.008219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.008219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.008219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.008219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.008886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.009124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.009308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] publish_errors = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.009486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.009664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.009848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.009848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.010209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.010394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.010627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.011065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.011249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.011434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.011434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.011795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.011978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.012162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] standard_threads = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.012345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.012345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.012736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.012917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.013125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.013349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.014058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.014244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.014441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] use_journal = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.014611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.014816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] use_json = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.014989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.015161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] use_stderr = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.015331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.015517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] use_syslog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.015700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.015700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] role.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.016121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:42:42.016305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] role.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.016517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.016727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.016727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.017083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.017272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.017272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.017751 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.017950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.018130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.018311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.018494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.018679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.018679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.019050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.019279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.019464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.019661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.019661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.020026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.020226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.020455 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.020643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.020643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.020991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.021168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.021379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.021571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.021759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.021949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.022132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.022317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.022317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.022676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.022875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.023067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.023257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.023257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.023621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.023804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.023997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.024189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.024189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.024589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.024780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.024780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.025135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.025323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.025514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.025817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.025817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.026161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.026344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.026529 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.026718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.026718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.027087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.027301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.027301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.027655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.027839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.028054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.028240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.028240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.028630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.028818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.029041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.029251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.029251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.029616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.029816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.030003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.030188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.030188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.030574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.030747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.030926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.031112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.031112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.031479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.031664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.031856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.032046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.032046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.032477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.032666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.032868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.033052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.033241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.033430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.033430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.033787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.033990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.034179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.034359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.034359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.034697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.034875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.034875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.035225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.035404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.035591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.035774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.035774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.036124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.036304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.036614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.036802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.036989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.037168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.037168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.037528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.037711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.037889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.038947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.040453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.040453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.040453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.040453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.040453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.040453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.040453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.041606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.041606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.041972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.042207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.042679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.043187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.043628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.044104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.045176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.045586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.045973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.046360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.046786 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.047211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.047630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.048078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.048551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.049043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.049490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.049926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.050372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.050817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.051269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.051773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.052249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.052760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.053211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.053673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.054142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.054583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.055047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.055515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.055957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.056475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.056849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.057310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.057745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.058203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.058645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.059130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.059561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.060044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.061136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.061590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.062019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.external = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.062465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.063038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.063483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.063931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.password = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.064352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.token = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.064804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.065265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.065726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.066176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.066658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.067122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.067576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.068037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.068538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.069002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.069470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.069963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.070433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.070870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.071060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.071269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.071456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.071639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.071827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.072245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.072746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.073294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.073771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.074256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.074706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.075210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.075652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.076683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.077156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.077626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.078052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.078534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.078986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.079475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.079931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.080438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.080913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.081388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.081840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.082027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.082314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.082780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.083240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.083693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.084153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.084641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.085091 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.085558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.086033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.086480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.086947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.087401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.087869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.088358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.088847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.089322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.089784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.090304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.090761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.091208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.091728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.092190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.092651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.093089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.093565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.094031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.094509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.094939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.095422 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.095901 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.096345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.096856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.097332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.097763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.098230 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.098671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.099128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.099581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.100069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.100548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.100999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.101472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.101959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.102410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.102853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.103330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.103798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.104256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.104742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.105188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.105622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.106097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.106596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.107550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.108037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.108557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.109038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.109511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.109909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.110349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.110785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.111240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.111713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.112187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.112677 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.113136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.113599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.114063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.114519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.115010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.115277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.115483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.115679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.115871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.116361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.116771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.117253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.117935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.118373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.118863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.119316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.119778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.120231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.120699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.121160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.121637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.122092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.122517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.123046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.123515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.123972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.124457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.124914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.125389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.125863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.126304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.126790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.127255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.127702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.128142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] role.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.128656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] role.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.129107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.129576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.130059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.130527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.131003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.131480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.131947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.132442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.132920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.133372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.133851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.134326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.134775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.135231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.136265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.136753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.137229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.137707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.138186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.139197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.139634 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.140131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.140639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.141089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.141526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.141976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.142428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.142910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.143402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.143914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.144319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.144794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.145240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.145691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.146141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.146622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.147096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.147590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.148051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.148556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.149029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.149485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.149964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.150424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.150886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.151333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.151750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.152193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.152621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.153062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.153539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.154142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.154568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.155030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.155476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.155927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.156435 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.156882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.157340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.157809 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.158277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.158721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.159181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.159645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.160121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.160602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.161069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.161523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.161959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.162357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:42:42.162633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 25 00:42:42.177792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.207160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.207483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.207815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.222737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.231718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.231973 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.259356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.259702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.338109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.398393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.398757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.399013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.399275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.399434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.399797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.400370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.400622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.425955 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.426316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.426526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.426746 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.426991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.427255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.427516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.439733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.453717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.454025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.454331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.454606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.454880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.455153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.455431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.474466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.474733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 25 00:42:42.475144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.475403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 25 00:42:42.475742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.476071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.476427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.476766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.487131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.487493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.487822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.488172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.495781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.499763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.500131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.500533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.500884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.501235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.501564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.501902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.502245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.502594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.502933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.503267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.527383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.529629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.529982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.535100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.535467 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.535809 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.542523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.542892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.549103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.549469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.549816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.550161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.550500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.550831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.551489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.557181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.557390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.557618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.557866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.558062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.558288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.568647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.569019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.572259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.572493 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.572753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.572976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.573304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.573457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.573603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.573603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.577041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.577393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.581691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.581864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.585100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.585350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.585350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.585612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.585753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.585941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.586152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.586366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 25 00:42:42.588977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.589359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.589705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.590066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.590407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.590757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.597588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.597849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 25 00:42:42.598131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.598340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 25 00:42:42.598615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.598879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.599137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.599416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.608569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.608948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.609278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.609475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.609628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.609797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.610045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.610329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.617323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.617881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.619148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.620869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.621056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.621230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.621350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.621632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.621931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.622228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.622492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.623299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.623596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.623868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.624151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.624462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.625114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.625469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.625835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.626170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.626623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.639457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.639833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.640174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.640564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.641023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.641371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.641720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.648125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.649783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.650085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.655676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.656035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.656409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.658644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.661810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.663518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.663883 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.664753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.666249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.667773 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.668068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.668461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.668612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.668890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.669234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.669688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.670020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.672262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.672666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.673036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.675145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.676686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.679661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.680072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.680352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.680648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.689269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.689571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.689856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.690307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.690666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.694936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.697038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.697341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.697621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.697893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.698213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.698485 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.698758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.698908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.699063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.699284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.699431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.699573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.699995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.700269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.700558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.700863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.704311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.707734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.712153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.712639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.712997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.713495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.713857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.714215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.731231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.731627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.731972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.732426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.736564 dv-cinder-master-3-smb devstack@keystone.service[81500]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55dcb714a9c0 pid: 81500 (default app) Sep 25 00:42:42.739750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.740323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9689a-6fef-424d-a465-52d442021b8c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:42.740641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9689a-6fef-424d-a465-52d442021b8c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:42.740916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9689a-6fef-424d-a465-52d442021b8c None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:42.741909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.743563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.743902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.745906 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Sun Sep 25 00:42:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:42:42.746745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.747034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.748836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.748836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.748836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.760479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.761041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.761329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.761619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.761901 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.762271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.762450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.778281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.782179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.785325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.786928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.788541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.788830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.789139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.796036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.797604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.800663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.800967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.801258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.801546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.809724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.810076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.810406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.817689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 25 00:42:42.817993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.818339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.818587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.818868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.819149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.819433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.819716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.820020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.820329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.820644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.820912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.821181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.821445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 25 00:42:42.861696 dv-cinder-master-3-smb devstack@keystone.service[81499]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55dcb714a9c0 pid: 81499 (default app) Sep 25 00:42:52.704298 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53962096-234e-4f8a-a98f-a3078e99569e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:52.705207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53962096-234e-4f8a-a98f-a3078e99569e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:52.705207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53962096-234e-4f8a-a98f-a3078e99569e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:52.707777 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 2/2] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:42:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:42:52.722627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:52.723627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:52.723627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:52.884076 dv-cinder-master-3-smb devstack@keystone.service[81499]: INFO dbcounter [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] Registered counter for database keystone Sep 25 00:42:52.924705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG oslo_db.sqlalchemy.engines [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81499) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:42:52.936912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writer thread running {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:42:53.281659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81499) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 25 00:42:53.282445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81499) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 25 00:42:53.353242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:42:53.355108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:42:53.589963 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1/3] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:42:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 874 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 25 00:42:54.160958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53962096-234e-4f8a-a98f-a3078e99569e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:42:54.289421 dv-cinder-master-3-smb devstack@keystone.service[81500]: INFO dbcounter [None req-53962096-234e-4f8a-a98f-a3078e99569e None None] Registered counter for database keystone Sep 25 00:42:54.313348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG oslo_db.sqlalchemy.engines [None req-53962096-234e-4f8a-a98f-a3078e99569e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81500) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:42:54.320634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writer thread running {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:42:54.458776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53962096-234e-4f8a-a98f-a3078e99569e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:42:54.463739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ab38b-8ede-4823-800b-75df1c3f964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:42:54.464506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ab38b-8ede-4823-800b-75df1c3f964f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:54.464839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ab38b-8ede-4823-800b-75df1c3f964f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:54.465165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ab38b-8ede-4823-800b-75df1c3f964f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:54.476748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ab38b-8ede-4823-800b-75df1c3f964f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:42:54.569185 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING keystone.common.rbac_enforcer.enforcer [None req-771ab38b-8ede-4823-800b-75df1c3f964f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 25 00:42:54.569763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ab38b-8ede-4823-800b-75df1c3f964f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:42:54.587749 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 3/4] 10.222.0.26 () {58 vars in 1324 bytes} [Sun Sep 25 00:42:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 00:42:54.595926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:42:54.617900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf55fa9-6a00-41f9-802b-8fcd3332c77a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:42:54.623749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd692509-e92f-4a98-acf9-1982f89903a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:42:54.625235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692509-e92f-4a98-acf9-1982f89903a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:54.625544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692509-e92f-4a98-acf9-1982f89903a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:54.625860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692509-e92f-4a98-acf9-1982f89903a8 admin admin] PATH_INFO: `/v3/projects/3708ba2402cc4ee7abd81255a285d2b9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:54.627780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd692509-e92f-4a98-acf9-1982f89903a8 admin admin] RBAC: Authorizing `identity:get_project(project_id=3708ba2402cc4ee7abd81255a285d2b9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:42:54.681300 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cd692509-e92f-4a98-acf9-1982f89903a8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 25 00:42:54.681636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd692509-e92f-4a98-acf9-1982f89903a8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:42:54.687004 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 2/5] 10.222.0.26 () {56 vars in 1207 bytes} [Sun Sep 25 00:42:54 2022] GET /identity/v3/projects/3708ba2402cc4ee7abd81255a285d2b9 => generated 344 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:42:57.014544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07f0e-8035-4eb0-8c7f-050dd08b3fe7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:57.015115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07f0e-8035-4eb0-8c7f-050dd08b3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:57.015536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07f0e-8035-4eb0-8c7f-050dd08b3fe7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:57.020640 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 4/6] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:42:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:42:57.027988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c67812-1da3-4fe2-a0b8-c4512911d03c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:57.029057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c67812-1da3-4fe2-a0b8-c4512911d03c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:57.029530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c67812-1da3-4fe2-a0b8-c4512911d03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:57.061584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f1c67812-1da3-4fe2-a0b8-c4512911d03c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:42:57.064216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f1c67812-1da3-4fe2-a0b8-c4512911d03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:42:57.078094 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 3/7] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:42:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 25 00:42:57.465586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad07f0e-8035-4eb0-8c7f-050dd08b3fe7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:42:57.479474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad07f0e-8035-4eb0-8c7f-050dd08b3fe7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:42:57.484163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260283a-d81a-49d6-a113-185098b7f223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:42:57.484815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260283a-d81a-49d6-a113-185098b7f223 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:57.485159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260283a-d81a-49d6-a113-185098b7f223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:57.485493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260283a-d81a-49d6-a113-185098b7f223 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:57.494708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260283a-d81a-49d6-a113-185098b7f223 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:42:57.496625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260283a-d81a-49d6-a113-185098b7f223 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:42:57.507578 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 5/8] 10.222.0.26 () {58 vars in 1324 bytes} [Sun Sep 25 00:42:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 25 00:42:57.514167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c67812-1da3-4fe2-a0b8-c4512911d03c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:42:57.524206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c67812-1da3-4fe2-a0b8-c4512911d03c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:42:57.529628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455fd42-3f11-4555-a144-0eeef2b2f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:42:57.530334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455fd42-3f11-4555-a144-0eeef2b2f79a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:42:57.530660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455fd42-3f11-4555-a144-0eeef2b2f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:42:57.530979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455fd42-3f11-4555-a144-0eeef2b2f79a admin admin] PATH_INFO: `/v3/users/7935e75bd19b46fd9203b785ff55ae84` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:42:57.533083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455fd42-3f11-4555-a144-0eeef2b2f79a admin admin] RBAC: Authorizing `identity:get_user(user_id=7935e75bd19b46fd9203b785ff55ae84)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:42:57.536725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455fd42-3f11-4555-a144-0eeef2b2f79a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:42:57.539827 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 4/9] 10.222.0.26 () {56 vars in 1198 bytes} [Sun Sep 25 00:42:57 2022] GET /identity/v3/users/7935e75bd19b46fd9203b785ff55ae84 => generated 247 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:00.206060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ee3a-6b44-420f-a994-12c37ef5a014 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:00.206941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ee3a-6b44-420f-a994-12c37ef5a014 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:00.206941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7ee3a-6b44-420f-a994-12c37ef5a014 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:00.208259 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 6/10] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:00.215879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db84dc-598c-48c9-b4ed-f815b6c1197d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:00.216687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db84dc-598c-48c9-b4ed-f815b6c1197d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:00.216687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db84dc-598c-48c9-b4ed-f815b6c1197d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:00.237812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d6db84dc-598c-48c9-b4ed-f815b6c1197d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:00.239082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d6db84dc-598c-48c9-b4ed-f815b6c1197d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:00.328454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94abf7f-6911-4e07-9d39-c592e29e11fa None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:00.328967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94abf7f-6911-4e07-9d39-c592e29e11fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:00.329450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94abf7f-6911-4e07-9d39-c592e29e11fa None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:00.331899 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 7/11] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:00.335148 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 5/12] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:00.341704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:00.342204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:00.342691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:00.430328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81500) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 25 00:43:00.431010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81500) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 25 00:43:00.486686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:00.489533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:00.616824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db84dc-598c-48c9-b4ed-f815b6c1197d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:00.643188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:00.671306 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 8/13] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:00.749166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db84dc-598c-48c9-b4ed-f815b6c1197d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:00.811624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:00.812488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:00.812876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:00.813260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:00.829867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:00.842735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:00.844532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:00.846811 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 6/14] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:43:00.852421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:00.946202 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:01.008627 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2a295-df0d-4fb3-bcd3-d172ffeb85c9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:01.012307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:01.075608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:01.077410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:01.077863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:01.078319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:01.079752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:01.080249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:01.081522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:01.082418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:01.102572 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 9/15] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:01.112497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:01.115504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d470e-fcc4-41e4-8fbe-493c28bcc2d1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:01.177657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943b239-432c-4191-9500-41ba56717368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:01.178461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943b239-432c-4191-9500-41ba56717368 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:01.178867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943b239-432c-4191-9500-41ba56717368 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:01.179220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943b239-432c-4191-9500-41ba56717368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:01.261438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943b239-432c-4191-9500-41ba56717368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:01.263000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943b239-432c-4191-9500-41ba56717368 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:01.283185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b3b7-1aa1-4417-bdf4-267dd58d3751 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:01.332582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:01.334420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:01.334873 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:01.335318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] PATH_INFO: `/v3/users/7935e75bd19b46fd9203b785ff55ae84` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:01.337857 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] RBAC: Authorizing `identity:get_user(user_id=7935e75bd19b46fd9203b785ff55ae84)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:01.341438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:01.344075 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 10/16] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:01 2022] GET /identity/v3/users/7935e75bd19b46fd9203b785ff55ae84 => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:01.350183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:01.377476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:01.389871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:01.402140 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 7/17] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 25 00:43:01.408642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943b239-432c-4191-9500-41ba56717368 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:01.475866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310493-0704-4976-9f08-00c4843f1f31 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:01.522137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:01.523391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:01.523874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:01.524331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:01.526502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:01.529757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:01.537966 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 11/18] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:01.539952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:01.546843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943b239-432c-4191-9500-41ba56717368 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:01.586196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:01.586851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:01.587135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:01.587410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:01.599550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:01.601121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:43:01.621144 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:01.622784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:43:01.624716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:43:01.624716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:43:01.624716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:01.624716 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 8/19] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:43:01.629359 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:01.680856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:01.703461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7090603-d9c2-467c-a2b3-b467272730de None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:01.705415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:01.757908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb87802-5184-4534-b9f0-d19370e37129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:01.758856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb87802-5184-4534-b9f0-d19370e37129 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:01.759269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb87802-5184-4534-b9f0-d19370e37129 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:01.759682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb87802-5184-4534-b9f0-d19370e37129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:01.761202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb87802-5184-4534-b9f0-d19370e37129 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.domain.id=default, user.id=7935e75bd19b46fd9203b785ff55ae84) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:01.761696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb87802-5184-4534-b9f0-d19370e37129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:01.764751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb87802-5184-4534-b9f0-d19370e37129 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:01.779841 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 12/20] 10.222.0.26 () {56 vars in 1322 bytes} [Sun Sep 25 00:43:01 2022] GET /identity/v3/role_assignments?user.id=7935e75bd19b46fd9203b785ff55ae84&scope.domain.id=default&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 233 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:01.780021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6787a-deaa-4a9d-9bec-69fa3eb84622 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:01.825784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947c0b3-73a8-423c-ae61-501ee03f7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:01.826445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947c0b3-73a8-423c-ae61-501ee03f7978 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:01.826725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947c0b3-73a8-423c-ae61-501ee03f7978 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:01.827000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947c0b3-73a8-423c-ae61-501ee03f7978 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:01.827712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947c0b3-73a8-423c-ae61-501ee03f7978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:01.828019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947c0b3-73a8-423c-ae61-501ee03f7978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:01.829607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947c0b3-73a8-423c-ae61-501ee03f7978 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:01.851871 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 9/21] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:05.623985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb12e8-7240-4a1d-9296-80a4af7b6f6b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:05.625428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb12e8-7240-4a1d-9296-80a4af7b6f6b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:05.626155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb12e8-7240-4a1d-9296-80a4af7b6f6b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:05.628881 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 13/22] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:05.646482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdaeb-48ab-4b4a-b270-39273cf3543b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:05.647955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdaeb-48ab-4b4a-b270-39273cf3543b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:05.648733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdaeb-48ab-4b4a-b270-39273cf3543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:05.694876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-349bdaeb-48ab-4b4a-b270-39273cf3543b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:05.697611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-349bdaeb-48ab-4b4a-b270-39273cf3543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:05.878812 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 10/23] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:06.173106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45059d-e1c8-4bbd-97a9-1eb5bddb6e5f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:06.174162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45059d-e1c8-4bbd-97a9-1eb5bddb6e5f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:06.174945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45059d-e1c8-4bbd-97a9-1eb5bddb6e5f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:06.177812 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 14/24] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:06.188161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd6afc-16f0-4955-839a-f32f016f40b0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:06.189340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd6afc-16f0-4955-839a-f32f016f40b0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:06.190092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd6afc-16f0-4955-839a-f32f016f40b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:06.233263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cefd6afc-16f0-4955-839a-f32f016f40b0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:06.236101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cefd6afc-16f0-4955-839a-f32f016f40b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:06.286684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:06.417418 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 11/25] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:06.442233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507937b9-47af-4530-b80e-baa508a380ae None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:06.442613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507937b9-47af-4530-b80e-baa508a380ae None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:06.443088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507937b9-47af-4530-b80e-baa508a380ae None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:06.445881 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 15/26] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:06.454884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5c032-d072-4cf4-bb6a-00ce3315955a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:06.455483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5c032-d072-4cf4-bb6a-00ce3315955a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:06.455966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5c032-d072-4cf4-bb6a-00ce3315955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:06.511097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-98f5c032-d072-4cf4-bb6a-00ce3315955a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:06.513969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-98f5c032-d072-4cf4-bb6a-00ce3315955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:06.648970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94633dcd-af24-4afa-b34e-01da2934719d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:06.649856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94633dcd-af24-4afa-b34e-01da2934719d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:06.649983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94633dcd-af24-4afa-b34e-01da2934719d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:06.652492 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 16/27] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:06.666174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c761e-c179-465b-9785-8952d7196f03 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:06.666671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c761e-c179-465b-9785-8952d7196f03 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:06.667162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c761e-c179-465b-9785-8952d7196f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:06.723534 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 12/28] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:06.725263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1d7c761e-c179-465b-9785-8952d7196f03 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:06.727423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1d7c761e-c179-465b-9785-8952d7196f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:06.919356 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 17/29] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 25 00:43:06.933401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5c032-d072-4cf4-bb6a-00ce3315955a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:07.106946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c761e-c179-465b-9785-8952d7196f03 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:07.159859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:07.204440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5c032-d072-4cf4-bb6a-00ce3315955a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:07.353734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:07.354848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:07.355335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:07.355890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:07.406111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:07.424191 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:07.425947 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:07.427476 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 13/30] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:43:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 494 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:07.445568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:07.484679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:07.516097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c761e-c179-465b-9785-8952d7196f03 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:07.655823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ece46-92bf-491c-be59-15358620728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:07.656970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ece46-92bf-491c-be59-15358620728d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:07.657448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ece46-92bf-491c-be59-15358620728d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:07.658533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ece46-92bf-491c-be59-15358620728d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:07.694998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4afa5-5237-4210-aac2-88bda71edd89 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:07.723523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ece46-92bf-491c-be59-15358620728d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:07.726672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ece46-92bf-491c-be59-15358620728d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-584ece46-92bf-491c-be59-15358620728d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:43:07.765243 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:07.767047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:43:07.768777 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:43:07.768777 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:43:07.768777 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:07.772038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:07.773177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:07.773347 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 18/31] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 668 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:07.773649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:07.774146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:07.775369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:07.775996 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:07.776482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ece46-92bf-491c-be59-15358620728d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:07.778732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:07.803547 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 14/32] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:07.804965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:07.821411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:08.000252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ece46-92bf-491c-be59-15358620728d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:08.020853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccf6f5-6f10-4aa5-82d5-450fe9ab763e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:08.074574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:08.076157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:08.077059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:08.077894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:08.079553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:08.082731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:08.093938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:08.095034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:08.095521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:08.096008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:08.098069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:08.101377 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:08.337814 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 15/33] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Sep 25 00:43:07 2022] POST /identity/v3/roles => generated 225 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:08.348872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:08.349182 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 19/34] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Sep 25 00:43:07 2022] POST /identity/v3/roles => generated 219 bytes in 567 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:08.349182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:08.396957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:08.539141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:08.635142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dbdb2-3f7e-4f1f-a2c8-897a52f28d0e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:08.712779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1634064-0a17-445e-95ee-0124a3402b2e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:08.729870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326d477-2bf5-40ba-841a-fd898c9d069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:08.731090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326d477-2bf5-40ba-841a-fd898c9d069a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:08.731567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326d477-2bf5-40ba-841a-fd898c9d069a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:08.732052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326d477-2bf5-40ba-841a-fd898c9d069a None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:08.733375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326d477-2bf5-40ba-841a-fd898c9d069a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:08.734759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326d477-2bf5-40ba-841a-fd898c9d069a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:08.737650 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326d477-2bf5-40ba-841a-fd898c9d069a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:08.770468 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 16/35] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:08.773166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e8e88-3d08-4e14-8a4a-be1fccba81a6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:08.774118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e8e88-3d08-4e14-8a4a-be1fccba81a6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:08.774500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e8e88-3d08-4e14-8a4a-be1fccba81a6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:08.777211 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 17/36] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:08.786409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e8e88-3d08-4e14-8a4a-be1fccba81a6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:08.803330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5b095-2275-49b4-8800-cb9ab726ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:08.805458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b095-2275-49b4-8800-cb9ab726ce9f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:08.805965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b095-2275-49b4-8800-cb9ab726ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:08.806466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b095-2275-49b4-8800-cb9ab726ce9f None admin] PATH_INFO: `/v3/users/7935e75bd19b46fd9203b785ff55ae84` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:08.810234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5b095-2275-49b4-8800-cb9ab726ce9f None admin] RBAC: Authorizing `identity:get_user(user_id=7935e75bd19b46fd9203b785ff55ae84)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:08.815636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5b095-2275-49b4-8800-cb9ab726ce9f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:08.817394 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 20/37] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:08 2022] GET /identity/v3/users/7935e75bd19b46fd9203b785ff55ae84 => generated 247 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:08.819885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fa4c-27c9-46fc-b740-497a263b131d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:08.820358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fa4c-27c9-46fc-b740-497a263b131d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:08.820891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fa4c-27c9-46fc-b740-497a263b131d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:08.867229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8941fa4c-27c9-46fc-b740-497a263b131d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:08.870816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8941fa4c-27c9-46fc-b740-497a263b131d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:09.028462 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 21/38] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:09.030060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941fa4c-27c9-46fc-b740-497a263b131d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:09.059256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:09.170020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e8e88-3d08-4e14-8a4a-be1fccba81a6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:09.226586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:09.244415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8e25f-fccf-4ac5-b3d8-a25a9216bfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:09.245466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e25f-fccf-4ac5-b3d8-a25a9216bfb5 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:09.245966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e25f-fccf-4ac5-b3d8-a25a9216bfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:09.246538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e25f-fccf-4ac5-b3d8-a25a9216bfb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:09.247740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8e25f-fccf-4ac5-b3d8-a25a9216bfb5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:09.250633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8e25f-fccf-4ac5-b3d8-a25a9216bfb5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:09.269694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941fa4c-27c9-46fc-b740-497a263b131d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:09.310904 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 18/39] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:43:08 2022] POST /identity/v3/projects => generated 301 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:09.313302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303c8b-cdc7-47d4-a68a-b75929294e90 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:09.313726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303c8b-cdc7-47d4-a68a-b75929294e90 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:09.314179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303c8b-cdc7-47d4-a68a-b75929294e90 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:09.320499 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 19/40] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:09.326771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b7f78-5efa-49f3-acd1-54af701f0db7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:09.327510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b7f78-5efa-49f3-acd1-54af701f0db7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:09.328111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b7f78-5efa-49f3-acd1-54af701f0db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:09.355601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:09.357719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:09.358551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:09.359488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:09.362385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:09.365741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:09.368536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e2b7f78-5efa-49f3-acd1-54af701f0db7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:09.369819 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 22/41] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:09.378061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e2b7f78-5efa-49f3-acd1-54af701f0db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:09.380105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:09.544080 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 20/42] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:09.557684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:09.557936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b7f78-5efa-49f3-acd1-54af701f0db7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:09.612077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca8ff2-cd86-4170-b607-e8c3c5ca2bcc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:09.689249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:09.690509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:09.691017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:09.691574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] PATH_INFO: `/v3/domains/default/users/7935e75bd19b46fd9203b785ff55ae84/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:09.695418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7935e75bd19b46fd9203b785ff55ae84, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:09.698289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:09.729228 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 23/43] 10.222.0.26 () {58 vars in 1361 bytes} [Sun Sep 25 00:43:09 2022] PUT /identity/v3/domains/default/users/7935e75bd19b46fd9203b785ff55ae84/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:43:09.785911 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b7f78-5efa-49f3-acd1-54af701f0db7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:09.808237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:09.882710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:09.883740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:09.884234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:09.884787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:09.886487 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:09.889954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:09.931258 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 21/44] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Sep 25 00:43:09 2022] POST /identity/v3/roles => generated 223 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:43:10.101687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:10.270326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b0ec4-4158-417f-b94c-4bc0793cd3bc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:10.328169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e14463-f7b7-490a-907a-11f7cb96b80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:10.329062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e14463-f7b7-490a-907a-11f7cb96b80d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:10.329463 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e14463-f7b7-490a-907a-11f7cb96b80d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:10.329858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e14463-f7b7-490a-907a-11f7cb96b80d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:10.331314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e14463-f7b7-490a-907a-11f7cb96b80d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:10.333028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e14463-f7b7-490a-907a-11f7cb96b80d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:10.334933 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 24/45] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:10.343249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:10.549148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e9112-bc2e-47a5-90c7-f8d72b128936 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:10.616326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aedb9-ac08-4d75-ba63-5cc534c25160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:10.617432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aedb9-ac08-4d75-ba63-5cc534c25160 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:10.617959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aedb9-ac08-4d75-ba63-5cc534c25160 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:10.618439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aedb9-ac08-4d75-ba63-5cc534c25160 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:10.619675 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5aedb9-ac08-4d75-ba63-5cc534c25160 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:10.622659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5aedb9-ac08-4d75-ba63-5cc534c25160 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:10.684427 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 22/46] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:43:10 2022] POST /identity/v3/projects => generated 312 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:12.782349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dce2ce-51c7-4823-9683-e733f3d8b50d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:12.783172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dce2ce-51c7-4823-9683-e733f3d8b50d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:12.783172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dce2ce-51c7-4823-9683-e733f3d8b50d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:12.784596 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 25/47] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:43:12.793620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea652c-3b6a-4c90-8272-f9e0acb187e3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:12.794402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea652c-3b6a-4c90-8272-f9e0acb187e3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:12.794402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea652c-3b6a-4c90-8272-f9e0acb187e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:12.816417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d4ea652c-3b6a-4c90-8272-f9e0acb187e3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:12.817905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d4ea652c-3b6a-4c90-8272-f9e0acb187e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:12.837563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:12.908966 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:13.404228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dce2ce-51c7-4823-9683-e733f3d8b50d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:13.409264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:13.611702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dce2ce-51c7-4823-9683-e733f3d8b50d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:13.684867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21548b-6d3b-475d-bdef-07a14a743fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:13.686019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21548b-6d3b-475d-bdef-07a14a743fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:13.686507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21548b-6d3b-475d-bdef-07a14a743fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:13.686983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21548b-6d3b-475d-bdef-07a14a743fe5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:13.712704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21548b-6d3b-475d-bdef-07a14a743fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1b21548b-6d3b-475d-bdef-07a14a743fe5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:13.725378 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:13.727306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:13.728894 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 26/49] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:43:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:13.738567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea652c-3b6a-4c90-8272-f9e0acb187e3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:13.827179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6b1a1-ccb9-4236-9a85-5b8902a336f5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:13.828039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6b1a1-ccb9-4236-9a85-5b8902a336f5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:13.828192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6b1a1-ccb9-4236-9a85-5b8902a336f5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:13.830952 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 27/50] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:43:13.840346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575aaf-7ced-42a2-885a-0ead275c29ca None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:13.840821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575aaf-7ced-42a2-885a-0ead275c29ca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:13.841255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f575aaf-7ced-42a2-885a-0ead275c29ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:13.873414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8f575aaf-7ced-42a2-885a-0ead275c29ca None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:13.875548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8f575aaf-7ced-42a2-885a-0ead275c29ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:13.952415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea652c-3b6a-4c90-8272-f9e0acb187e3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:13.997135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:14.017646 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 28/51] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:14.021603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:14.023108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:14.023571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:14.024041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:14.025342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:14.025879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:14.029248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:14.046659 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 24/52] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:14.056587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f575aaf-7ced-42a2-885a-0ead275c29ca None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:14.253623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f575aaf-7ced-42a2-885a-0ead275c29ca None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:14.327974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:14.329157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:14.329650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:14.330117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] PATH_INFO: `/v3/users/7935e75bd19b46fd9203b785ff55ae84` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:14.332546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] RBAC: Authorizing `identity:get_user(user_id=7935e75bd19b46fd9203b785ff55ae84)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:14.335577 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:14.338769 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 29/53] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:14 2022] GET /identity/v3/users/7935e75bd19b46fd9203b785ff55ae84 => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:14.348557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:14.379855 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:14.525010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:14.563875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1ab2e-aa77-4835-bc57-7ab71c17bacd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:14.631953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:14.633095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:14.633578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:14.634060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:14.636543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:14.639907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:14.643001 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 25/54] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:14.645304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:14.653471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:14.732711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44dc611-702e-46f3-a8de-b2d872b4b07b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:14.802003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:14.803004 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:14.803473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:14.803953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:14.806036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:14.808664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:14.811425 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 30/55] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:14.818554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:14.820673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:14.860418 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61caac2b-f88a-44dc-a1ec-c5d404ab4176 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:14.927983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e39f0-6df3-4723-97fc-84df57237003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:14.929087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e39f0-6df3-4723-97fc-84df57237003 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:14.929585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e39f0-6df3-4723-97fc-84df57237003 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:14.930058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e39f0-6df3-4723-97fc-84df57237003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:14.931410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e39f0-6df3-4723-97fc-84df57237003 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.domain.id=default, user.id=7935e75bd19b46fd9203b785ff55ae84) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:14.931973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e39f0-6df3-4723-97fc-84df57237003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:14.935365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e39f0-6df3-4723-97fc-84df57237003 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:14.949436 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 26/56] 10.222.0.26 () {56 vars in 1322 bytes} [Sun Sep 25 00:43:14 2022] GET /identity/v3/role_assignments?user.id=7935e75bd19b46fd9203b785ff55ae84&scope.domain.id=default&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 530 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:15.025061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da35319-bcae-4032-b30f-ee51f782d212 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:15.083208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:15.102822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb771c1-0c7f-4491-bbe7-97fd7d577d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:15.103888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb771c1-0c7f-4491-bbe7-97fd7d577d9c None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:15.104367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb771c1-0c7f-4491-bbe7-97fd7d577d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:15.104876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb771c1-0c7f-4491-bbe7-97fd7d577d9c None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:15.106157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb771c1-0c7f-4491-bbe7-97fd7d577d9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:15.108991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb771c1-0c7f-4491-bbe7-97fd7d577d9c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:15.169232 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 31/57] 10.222.0.26 () {60 vars in 1140 bytes} [Sun Sep 25 00:43:14 2022] POST /identity/v3/projects => generated 298 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:43:18.992612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2688b4-ff67-45ee-8541-951ed62e5b97 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:18.993505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2688b4-ff67-45ee-8541-951ed62e5b97 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:18.993505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2688b4-ff67-45ee-8541-951ed62e5b97 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:18.995053 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 27/58] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:19.005117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863783e3-3e05-4a8c-a66a-7d6e903be538 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:19.005714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863783e3-3e05-4a8c-a66a-7d6e903be538 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:19.006017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863783e3-3e05-4a8c-a66a-7d6e903be538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:19.037057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-863783e3-3e05-4a8c-a66a-7d6e903be538 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:19.039177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-863783e3-3e05-4a8c-a66a-7d6e903be538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:19.282219 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 32/59] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:19.567535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2688b4-ff67-45ee-8541-951ed62e5b97 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:19.703326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2688b4-ff67-45ee-8541-951ed62e5b97 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:19.753194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53f71-3d4d-4159-9ffc-9bc77ca20bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:19.753960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53f71-3d4d-4159-9ffc-9bc77ca20bad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:19.754296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53f71-3d4d-4159-9ffc-9bc77ca20bad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:19.754762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53f71-3d4d-4159-9ffc-9bc77ca20bad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:19.756897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53f71-3d4d-4159-9ffc-9bc77ca20bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:19.759360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53f71-3d4d-4159-9ffc-9bc77ca20bad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:19.762765 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 28/60] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:19.771395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863783e3-3e05-4a8c-a66a-7d6e903be538 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:19.941379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863783e3-3e05-4a8c-a66a-7d6e903be538 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:19.953916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:19.985702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecb8cb-b83b-40cc-bb31-2d40c1e8bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:19.986488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecb8cb-b83b-40cc-bb31-2d40c1e8bb75 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:19.986965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecb8cb-b83b-40cc-bb31-2d40c1e8bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:19.987432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecb8cb-b83b-40cc-bb31-2d40c1e8bb75 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:19.988311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecb8cb-b83b-40cc-bb31-2d40c1e8bb75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:19.990149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecb8cb-b83b-40cc-bb31-2d40c1e8bb75 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:20.026002 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 33/61] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:43:19 2022] POST /identity/v3/users => generated 275 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:25.341802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf642321-a75c-494c-a133-10f9815c4f9a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:25.341802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf642321-a75c-494c-a133-10f9815c4f9a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:25.341802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf642321-a75c-494c-a133-10f9815c4f9a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:25.343210 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 29/62] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:25.362098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefffd2-4259-4d19-8928-f1dd29c7e1b0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:25.362908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefffd2-4259-4d19-8928-f1dd29c7e1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:25.363047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefffd2-4259-4d19-8928-f1dd29c7e1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:25.409520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2fefffd2-4259-4d19-8928-f1dd29c7e1b0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:25.411675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2fefffd2-4259-4d19-8928-f1dd29c7e1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:25.583712 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 34/63] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:25.667219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ff56-cc30-4fa8-8706-5292b851c57b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:25.668250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ff56-cc30-4fa8-8706-5292b851c57b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:25.669049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ff56-cc30-4fa8-8706-5292b851c57b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:25.671804 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 30/64] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:25.681354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85060-4f6d-4ed1-817e-41f2e38e2686 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:25.681862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85060-4f6d-4ed1-817e-41f2e38e2686 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:25.682362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b85060-4f6d-4ed1-817e-41f2e38e2686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:25.730860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-97b85060-4f6d-4ed1-817e-41f2e38e2686 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:25.733380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-97b85060-4f6d-4ed1-817e-41f2e38e2686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:25.894402 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 35/65] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:26.151868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60d9c0-6fcd-40b7-a1c3-91a7ff83a3e5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:26.153528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60d9c0-6fcd-40b7-a1c3-91a7ff83a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:26.154320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60d9c0-6fcd-40b7-a1c3-91a7ff83a3e5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:26.157389 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 31/66] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:26.167476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9863a-fb3e-4b56-bd27-92edb47ff88f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:26.168622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9863a-fb3e-4b56-bd27-92edb47ff88f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:26.169481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9863a-fb3e-4b56-bd27-92edb47ff88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:26.214323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6dc9863a-fb3e-4b56-bd27-92edb47ff88f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:26.217294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6dc9863a-fb3e-4b56-bd27-92edb47ff88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:26.273881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:26.357932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60d9c0-6fcd-40b7-a1c3-91a7ff83a3e5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:26.408527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:26.413887 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 36/67] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:26.465695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3888-44a2-41da-a534-c9c6fa69ceb4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:26.466664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3888-44a2-41da-a534-c9c6fa69ceb4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:26.467432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3888-44a2-41da-a534-c9c6fa69ceb4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:26.474644 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 37/68] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:43:26.483776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaef1d-746e-471f-8a56-11f859f20492 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:26.484731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaef1d-746e-471f-8a56-11f859f20492 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:26.485521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaef1d-746e-471f-8a56-11f859f20492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:26.544537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5dbaef1d-746e-471f-8a56-11f859f20492 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:26.547206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5dbaef1d-746e-471f-8a56-11f859f20492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:26.604983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60d9c0-6fcd-40b7-a1c3-91a7ff83a3e5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:26.666596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:26.667817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:26.668344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:26.668950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:26.685193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:26.691683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:26.693075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:26.694665 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:26.696963 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 32/69] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:43:26.697262 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 38/70] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:26.699394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2dee-6faf-4b3b-ad37-c49c20f4e513 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:26.699900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2dee-6faf-4b3b-ad37-c49c20f4e513 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:26.700431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2dee-6faf-4b3b-ad37-c49c20f4e513 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:26.702760 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 39/71] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:26.703254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:26.706026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e2dee-6faf-4b3b-ad37-c49c20f4e513 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:26.872699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:26.897899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:26.933952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e2dee-6faf-4b3b-ad37-c49c20f4e513 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:26.952460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d24346-6375-4eeb-abb9-6c48b679c83e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:26.994872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb1131-6745-473f-b7ec-010a6555c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:26.996648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb1131-6745-473f-b7ec-010a6555c446 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:26.997234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb1131-6745-473f-b7ec-010a6555c446 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:26.998092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb1131-6745-473f-b7ec-010a6555c446 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:26.999076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb1131-6745-473f-b7ec-010a6555c446 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:26.999589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb1131-6745-473f-b7ec-010a6555c446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:27.002275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb1131-6745-473f-b7ec-010a6555c446 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:27.015126 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 40/72] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:27.015769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:27.017037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:27.017514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:27.017997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:27.020203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada3155-0253-4cbd-8502-ca910423f659 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:27.021100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada3155-0253-4cbd-8502-ca910423f659 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:27.022551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada3155-0253-4cbd-8502-ca910423f659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:27.039344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:27.047581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fada3155-0253-4cbd-8502-ca910423f659 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:27.049397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fada3155-0253-4cbd-8502-ca910423f659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:27.050211 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:27.051097 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:27.051967 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:27.061852 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 33/73] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:43:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:27.061852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:27.154568 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 41/74] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:27.156011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada3155-0253-4cbd-8502-ca910423f659 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:27.256808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beaecf-d93d-4886-a6aa-33e53a2f0b29 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:27.304636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:27.323590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:27.339802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:27.340937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:27.341427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:27.341924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:27.363388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada3155-0253-4cbd-8502-ca910423f659 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:27.368702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:27.379021 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:27.380749 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 34/75] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:43:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:27.386085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:27.445261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:27.446369 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:27.446803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:27.447250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:27.481725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:27.484242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:27.487380 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 42/76] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:27 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:27.488851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:27.619397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4fcec-71fb-4ebf-83b5-9c82c9c179bd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:27.685518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:27.686588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:27.687094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:27.687594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:27.688367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979fd9f-0e95-40ef-8638-15e11b0c0ad6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:27.688912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:27.689469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:27.692265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:27.709014 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 35/77] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:27.710576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:27.754242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:27.755387 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:27.755888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:27.756424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:27.758439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:27.761470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:27.764565 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 43/78] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:27.765916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:27.776459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:27.836407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:27.927923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156817d-ad01-4250-8708-ab7993a7e13f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:27.975324 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6729c-0035-4fee-9767-8f0ebe472b1b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:27.993415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:27.994445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:27.994919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:27.995389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:27.996724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:27.997203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:27.999837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:28.016228 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 36/79] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:43:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:28.017782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:28.040985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:28.041946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:28.042440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:28.042965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:28.201322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:28.202856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:28.207688 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:28.217475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bd2de-ed5d-4e24-9740-e074e6f4df58 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:28.249849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:28.283573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:28.284699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:28.285190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:28.285667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:28.303779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:28.316260 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:28.317762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:28.318568 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 37/80] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:28.320369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:28.377960 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 44/81] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:28.378838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:28.521783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee150b-3cb4-4e05-98bd-6da00a6e402d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:28.561817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5230d9-ea90-49ad-b92f-84e73284ec1f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:28.566642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:28.590282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:28.591387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:28.591898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:28.592414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] PATH_INFO: `/v3/users/7935e75bd19b46fd9203b785ff55ae84` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:28.594616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] RBAC: Authorizing `identity:get_user(user_id=7935e75bd19b46fd9203b785ff55ae84)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:28.597566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:28.600733 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 38/82] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:28 2022] GET /identity/v3/users/7935e75bd19b46fd9203b785ff55ae84 => generated 247 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:28.602251 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:28.634624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:28.635762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:28.636265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:28.636757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:28.637848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:28.640497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:28.672656 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 45/83] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:43:28 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:28.673834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:28.700680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:28.815367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb6a54-7594-460c-a415-59e50f84ea09 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:28.849644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c03516-091d-4eed-8022-b12a10c7456c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:28.867191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:28.868079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:28.868628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:28.869084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:28.870834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:28.872804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:28.874889 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 39/84] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:28 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:28.876443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:28.930717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:28.932550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:28.933878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:28.934657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:28.935980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:28.936538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:28.939265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:28.957598 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 46/85] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:28.959065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:29.017695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:29.071411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048f857-aee3-4cea-9223-1452fbe27c44 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:29.096076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:29.141782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:29.143309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:29.143937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:29.144676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:29.146700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:29.150220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:29.157503 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 40/86] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:28 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:29.158699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:29.206783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab42df7-6a50-44ef-8c8c-96213c38f8e8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:29.283638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:29.285276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:29.286185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:29.286960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:29.405375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfe194-78fb-4499-9d7d-a864e3b5cc85 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:29.465501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:29.469743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:29.489579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:29.490641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:29.491125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:29.491621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:29.573184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:29.611845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:29.655245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:29.658349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:29.721877 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 47/87] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:29.723484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:29.893471 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 41/88] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:29.895207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:29.942213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba46b0-c836-42a4-bf2d-fb982e8fcdf0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:29.989699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:30.013026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:30.014102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:30.014599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:30.015093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:30.017254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:30.020032 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:30.023480 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 48/89] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:29 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:30.025753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:30.064857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:30.109707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8daf8f-5070-452a-91d6-92998c74bae1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:30.185983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:30.187143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:30.187649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:30.188141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:30.189545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=f8ff084606084f0fb810529a2bea08e9) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:30.190183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:30.193176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:30.204983 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 42/90] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:29 2022] GET /identity/v3/role_assignments?user.id=f8ff084606084f0fb810529a2bea08e9&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=896da438cb834959bd42187c944ea2f7 => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:30.211799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:30.225771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62228585-c968-4684-9b96-724a1bd6e3f1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:30.320447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:30.320447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:30.320447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:30.320447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:30.334096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:30.334096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:30.334096 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 49/91] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:30 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:30.340959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:30.578580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dace4f0-a2eb-435d-a7ef-b6f4665e29e7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:30.608329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:30.708363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:30.710593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:30.711397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:30.712015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b5b27-19f8-428a-af9c-ff53d9dabbcc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:30.712491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:30.721639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:30.724927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:30.728624 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 43/92] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:30 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:30.736415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:30.772011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:30.851477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:30.853069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:30.853683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:30.854185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:31.011880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:31.015429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:31.058137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb50e3e-45aa-4f9d-891d-746e2d3328b5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:31.143095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:31.146227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:31.155104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:31.156000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:31.157754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=7935e75bd19b46fd9203b785ff55ae84) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:31.158645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:31.161666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:31.187491 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 44/93] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:30 2022] GET /identity/v3/role_assignments?user.id=7935e75bd19b46fd9203b785ff55ae84&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:31.189577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:31.247854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:31.406999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:31.448419 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 50/94] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1107 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:31.454106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:31.482423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477895df-cbbd-474a-9778-13a14567b547 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:31.567736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:31.569408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:31.570170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:31.570933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:31.572647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] RBAC: Adding query filter params (role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=f8ff084606084f0fb810529a2bea08e9) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:31.573557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:31.577153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:31.596616 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 45/95] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:31 2022] GET /identity/v3/role_assignments?user.id=f8ff084606084f0fb810529a2bea08e9&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 259 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:31.694823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805601a7-8e01-4804-9054-b05d7752230b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:31.777255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d7e3f-ebb3-47c6-b986-5cfcb3ebaa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:31.778478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7e3f-ebb3-47c6-b986-5cfcb3ebaa71 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:31.779003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7e3f-ebb3-47c6-b986-5cfcb3ebaa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:31.779515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7e3f-ebb3-47c6-b986-5cfcb3ebaa71 None admin] PATH_INFO: `/v3/projects/4882fc69f4c84744810e6cdfcbfc16df` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:31.783414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d7e3f-ebb3-47c6-b986-5cfcb3ebaa71 None admin] RBAC: Authorizing `identity:get_project(project_id=4882fc69f4c84744810e6cdfcbfc16df)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:31.786357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d7e3f-ebb3-47c6-b986-5cfcb3ebaa71 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:31.790219 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 51/96] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:31 2022] GET /identity/v3/projects/4882fc69f4c84744810e6cdfcbfc16df => generated 312 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:31.800186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:32.018361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4f1e-a848-47d2-a691-7358c6c83a37 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:32.089990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91504790-58ec-45ab-9f69-913af356fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:32.091616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504790-58ec-45ab-9f69-913af356fe93 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:32.092605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504790-58ec-45ab-9f69-913af356fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:32.093357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504790-58ec-45ab-9f69-913af356fe93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:32.095038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91504790-58ec-45ab-9f69-913af356fe93 None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=4882fc69f4c84744810e6cdfcbfc16df, user.id=f8ff084606084f0fb810529a2bea08e9) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:32.095889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91504790-58ec-45ab-9f69-913af356fe93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:32.099011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91504790-58ec-45ab-9f69-913af356fe93 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:32.110859 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 46/97] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:31 2022] GET /identity/v3/role_assignments?user.id=f8ff084606084f0fb810529a2bea08e9&scope.project.id=4882fc69f4c84744810e6cdfcbfc16df&role.id=896da438cb834959bd42187c944ea2f7 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:33.262051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a367e-9e22-4d77-b08f-2fdfc6022a98 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:33.262829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a367e-9e22-4d77-b08f-2fdfc6022a98 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:33.263003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a367e-9e22-4d77-b08f-2fdfc6022a98 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:33.265651 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 52/98] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:33.275019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0520eb3-c88e-454a-a8f3-00105f6af1d6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:33.275704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0520eb3-c88e-454a-a8f3-00105f6af1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:33.276042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0520eb3-c88e-454a-a8f3-00105f6af1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:33.298692 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:33.322534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b0520eb3-c88e-454a-a8f3-00105f6af1d6 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:33.325291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b0520eb3-c88e-454a-a8f3-00105f6af1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:33.500029 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 47/99] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:34.153895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a367e-9e22-4d77-b08f-2fdfc6022a98 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:34.222209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:34.446092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a367e-9e22-4d77-b08f-2fdfc6022a98 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:34.540174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:34.541431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:34.541949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:34.542983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:34.575431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26e02a-37d0-4d40-85da-9ff1b36013f7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:34.577041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26e02a-37d0-4d40-85da-9ff1b36013f7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:34.577878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26e02a-37d0-4d40-85da-9ff1b36013f7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:34.578035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:34.585059 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 48/100] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:34.588590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:34.590299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:43:34.591781 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:34.595600 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 53/101] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:34.598606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d040664-9907-4a6a-9ad3-0b0042003771 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:34.599528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d040664-9907-4a6a-9ad3-0b0042003771 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:34.600366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d040664-9907-4a6a-9ad3-0b0042003771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:34.602741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:34.654831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2d040664-9907-4a6a-9ad3-0b0042003771 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:34.657559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2d040664-9907-4a6a-9ad3-0b0042003771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:34.859499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:34.900496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088489-b6a4-4649-b9ab-b2cec1fadce0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:34.946629 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 49/102] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:34.998914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:34.999963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:35.000492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:35.000983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:35.002312 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:35.002899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:35.005349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:35.026580 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 54/103] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:35.036515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d040664-9907-4a6a-9ad3-0b0042003771 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:35.209348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:35.288489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d040664-9907-4a6a-9ad3-0b0042003771 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:35.361561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:35.362720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:35.363222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:35.363792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] PATH_INFO: `/v3/users/c21c9bc05bc340358a9ecde27feaeeaf` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:35.390431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] RBAC: Authorizing `identity:get_user(user_id=c21c9bc05bc340358a9ecde27feaeeaf)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:35.393173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:35.395927 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 50/104] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:35 2022] GET /identity/v3/users/c21c9bc05bc340358a9ecde27feaeeaf => generated 289 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:35.403820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:35.598394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede823ab-51b9-4c07-8ee2-fad22483235e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:35.681089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71860046-e462-4bb6-9574-08a0a1595e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:35.682164 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71860046-e462-4bb6-9574-08a0a1595e3d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:35.682475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71860046-e462-4bb6-9574-08a0a1595e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:35.682760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71860046-e462-4bb6-9574-08a0a1595e3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:35.729949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:35.783312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:35.842597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71860046-e462-4bb6-9574-08a0a1595e3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:35.845429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71860046-e462-4bb6-9574-08a0a1595e3d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:36.076848 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 55/105] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:36.083756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de06c1-e455-484b-88e2-a0ae0965cebf None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:36.084780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de06c1-e455-484b-88e2-a0ae0965cebf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:36.085562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de06c1-e455-484b-88e2-a0ae0965cebf None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:36.088099 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 56/106] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:36.095234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de06c1-e455-484b-88e2-a0ae0965cebf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:36.108433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf362a-5025-4472-bbbe-3a9015e72be4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:36.190983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:36.241123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b3518-508c-4009-b403-f3187232952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:36.242173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b3518-508c-4009-b403-f3187232952c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:36.242645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b3518-508c-4009-b403-f3187232952c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:36.243103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b3518-508c-4009-b403-f3187232952c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:36.247884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b3518-508c-4009-b403-f3187232952c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:36.250604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b3518-508c-4009-b403-f3187232952c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:36.258762 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 51/107] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:36.267930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1580bb-4167-4441-8892-91281adff35c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:36.268444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1580bb-4167-4441-8892-91281adff35c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:36.268857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1580bb-4167-4441-8892-91281adff35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:36.273690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:36.326642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1e1580bb-4167-4441-8892-91281adff35c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:36.328609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1e1580bb-4167-4441-8892-91281adff35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:36.425636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de06c1-e455-484b-88e2-a0ae0965cebf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:36.492345 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 52/108] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:36.500434 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1580bb-4167-4441-8892-91281adff35c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:36.548457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:36.549502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:36.549935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:36.550352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:36.553266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:36.555142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:36.557115 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 57/109] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:36 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:36.564425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:36.724795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1580bb-4167-4441-8892-91281adff35c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:36.777560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:36.830923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f703cca-3fbe-4233-afd4-6ea860c70734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:36.833162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703cca-3fbe-4233-afd4-6ea860c70734 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:36.833987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703cca-3fbe-4233-afd4-6ea860c70734 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:36.834780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703cca-3fbe-4233-afd4-6ea860c70734 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:36.836295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f703cca-3fbe-4233-afd4-6ea860c70734 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:36.840036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f703cca-3fbe-4233-afd4-6ea860c70734 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:36.885796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a56d2f-3551-4e66-8431-1477fda625ca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:36.904592 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 53/110] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:43:36 2022] POST /identity/v3/projects => generated 302 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:36.909555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23e33c-7eb1-42ca-8588-168f0d170dfb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:36.913218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23e33c-7eb1-42ca-8588-168f0d170dfb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:36.914154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23e33c-7eb1-42ca-8588-168f0d170dfb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:36.919674 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 54/111] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:36.928720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001822cf-4a7d-4227-a225-1b71f99b0d2c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:36.929771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001822cf-4a7d-4227-a225-1b71f99b0d2c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:36.930547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001822cf-4a7d-4227-a225-1b71f99b0d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:36.963644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:36.981671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-001822cf-4a7d-4227-a225-1b71f99b0d2c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:36.985894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-001822cf-4a7d-4227-a225-1b71f99b0d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:37.040461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:37.041660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:37.042818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:37.043252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:37.044168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] RBAC: Adding query filter params (role.id=fef78cf3dcc64b77a03f0b53167b7430, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=c21c9bc05bc340358a9ecde27feaeeaf) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:37.044618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:37.046436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:37.063941 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 58/112] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:36 2022] GET /identity/v3/role_assignments?user.id=c21c9bc05bc340358a9ecde27feaeeaf&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=fef78cf3dcc64b77a03f0b53167b7430 => generated 259 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:37.068557 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:37.168740 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 55/113] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:37.171633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd67370-783f-4998-a638-9a8a2a267eec None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:37.180425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd67370-783f-4998-a638-9a8a2a267eec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:37.181579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd67370-783f-4998-a638-9a8a2a267eec None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:37.184647 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 56/114] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:37 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:37.202224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03731d3c-49bd-493c-8fe6-66b70d347abb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:37.202801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03731d3c-49bd-493c-8fe6-66b70d347abb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:37.203257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03731d3c-49bd-493c-8fe6-66b70d347abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:37.266419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-03731d3c-49bd-493c-8fe6-66b70d347abb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:37.268993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03731d3c-49bd-493c-8fe6-66b70d347abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:37.374334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869601-97b3-4784-9cf4-fa2ab0b4b3fc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:37.500678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:37.501903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:37.502469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:37.502961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:37.518726 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 57/115] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:37.521282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2672e9-31e1-4f3d-97e5-92be15a9c1ca None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:37.522049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2672e9-31e1-4f3d-97e5-92be15a9c1ca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:37.522545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2672e9-31e1-4f3d-97e5-92be15a9c1ca None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:37.525123 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 58/116] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:37.530232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:37.535657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dd241-df0f-40e3-8b49-61b17d643219 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:37.536158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dd241-df0f-40e3-8b49-61b17d643219 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:37.536734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dd241-df0f-40e3-8b49-61b17d643219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:37.546979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:37.548698 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:37.552136 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 59/117] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 485 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:37.559667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:37.574299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ea9dd241-df0f-40e3-8b49-61b17d643219 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:37.576083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ea9dd241-df0f-40e3-8b49-61b17d643219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:37.670441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:37.692205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:37.797149 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 59/118] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:37.850182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9dd241-df0f-40e3-8b49-61b17d643219 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:37.911245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3ad63-27cb-4313-85e2-f1a1da1dcb42 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:38.015587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:38.016906 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:38.017374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:38.017859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:38.019180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:38.019733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:38.022855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:38.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 60/119] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 497 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:38.064278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:38.228661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9dd241-df0f-40e3-8b49-61b17d643219 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:38.328174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:38.329325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:38.329873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:38.330349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:38.362367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:38.368894 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:38.369903 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:38.370855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:38.374229 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 60/120] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:43:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 530 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:43:38.375470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:38.420574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:38.447671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2870fbe-4bf4-4996-a09c-7dcd81686ea1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:38.451351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:38.537165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:38.538327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:38.538830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:38.539322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:38.541719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:38.545061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:38.547902 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 61/121] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:38 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:38.549568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:38.561614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774162d5-bbd9-48e6-b521-d0993277efb6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:38.628180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:38.629297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:38.629798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:38.630291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:38.651341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:38.660999 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:38.662672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:38.664123 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 61/122] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:43:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:38.664901 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:38.770585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829aac2a-0e76-4c6b-b205-1ee260674ea6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:38.829295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:38.836215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:38.838101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:38.838794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:38.839832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:38.841020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:38.841685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:38.845181 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:38.861595 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 62/123] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:38.862940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:38.867126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5597eea-9a3b-49fe-b3d2-ced5d49cc528 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:38.925513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:38.929926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:38.931345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:38.932685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:38.933898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:38.956860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:38.965422 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:38.966964 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:38.968171 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:38.971007 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 62/124] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:38.972359 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:39.084515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a802cb-263c-4af5-85f7-c143f6f50825 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:39.155692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:39.157872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:39.158456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:39.159188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:39.161737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:39.166368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:39.171461 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 63/125] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:38 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:39.173516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:39.185725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dfd54-e856-4875-8fc3-43029bdfd58e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:39.250465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:39.251537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:39.251991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:39.252543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:39.253804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:39.254322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:39.256031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:39.267337 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 63/126] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:43:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:39.268592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:39.273872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:39.399162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:39.407329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec482c38-a4cd-4b83-824d-fcc776b4f86e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:39.439152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89279c8-a049-4f00-a43e-5dfdcab6af31 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:39.479498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:39.480827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:39.481338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:39.481791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] PATH_INFO: `/v3/users/7935e75bd19b46fd9203b785ff55ae84` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:39.484142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] RBAC: Authorizing `identity:get_user(user_id=7935e75bd19b46fd9203b785ff55ae84)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:39.487333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:39.489717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:39.490353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:39.490894 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 64/127] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:39 2022] GET /identity/v3/users/7935e75bd19b46fd9203b785ff55ae84 => generated 247 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:39.491005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:39.492062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:39.497043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:39.498087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:39.498410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:39.500765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:39.514604 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 64/128] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:39.516230 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:39.649025 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:39.713568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d01f48-2449-4234-82a4-d8fed20e9991 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:39.719570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70951a4-2f0c-4e37-9627-d8fd32cd245c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:39.780312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:39.781581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:39.782582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:39.783156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:39.786588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:39.791205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:39.795533 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 65/129] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:39 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:39.797460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:39.799524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:39.800921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:39.801548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:39.802227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/f8ff084606084f0fb810529a2bea08e9/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:39.806453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] RBAC: Authorizing `identity:create_grant(project_id=4010acd13d9445b1943eac27cb61aaa2, user_id=f8ff084606084f0fb810529a2bea08e9, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:39.808724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:39.838748 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 65/130] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:43:39 2022] PUT /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/f8ff084606084f0fb810529a2bea08e9/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:43:39.840530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:39.946521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:40.024660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fa8b2-a4d4-4ff5-a49d-8ab1e9831dd3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:40.091145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:40.092945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:40.093808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:40.094612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:40.097232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:40.100580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:40.103996 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 66/131] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:39 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:40.106290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:40.112225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:40.175618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96788467-0f5b-4049-a95a-b2e0a213c81b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:40.263082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:40.265348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:40.266514 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:40.267285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:40.270035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:40.280655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:40.286399 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 66/132] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:39 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:40.288352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:40.328866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c40d5-879b-44bd-a896-be825e35974d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:40.399778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:40.400754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:40.401150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:40.402077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:40.403884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:40.406173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:40.409267 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 67/133] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:40 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:40.410979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:40.479520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:40.498082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6319-220c-44ef-ad1f-2ac536d5bd31 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:40.596435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:40.597531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:40.598000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:40.602506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/7935e75bd19b46fd9203b785ff55ae84/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:40.610759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] RBAC: Authorizing `identity:create_grant(project_id=4010acd13d9445b1943eac27cb61aaa2, user_id=7935e75bd19b46fd9203b785ff55ae84, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:40.617873 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:40.639695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:40.673326 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 67/134] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:43:40 2022] PUT /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/7935e75bd19b46fd9203b785ff55ae84/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:43:40.680751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:40.706952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49327b8d-3548-4f1a-adf0-a5d02d22ac88 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:40.799194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:40.800876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:40.801645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:40.802437 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] PATH_INFO: `/v3/projects/4882fc69f4c84744810e6cdfcbfc16df` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:40.808844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] RBAC: Authorizing `identity:get_project(project_id=4882fc69f4c84744810e6cdfcbfc16df)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:40.812227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:40.819691 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 68/135] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:40 2022] GET /identity/v3/projects/4882fc69f4c84744810e6cdfcbfc16df => generated 312 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:40.825871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:41.044197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38154c6-3ace-4adb-8e48-c3c0cb086f72 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:41.113138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cae458-a7c5-41ea-8188-738512c3251a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:41.114045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cae458-a7c5-41ea-8188-738512c3251a None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:41.114344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cae458-a7c5-41ea-8188-738512c3251a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:41.114620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cae458-a7c5-41ea-8188-738512c3251a None admin] PATH_INFO: `/v3/projects/4882fc69f4c84744810e6cdfcbfc16df/users/f8ff084606084f0fb810529a2bea08e9/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:41.119647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deece480-c193-44bd-af5e-abc1caf3bc3e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:41.122580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cae458-a7c5-41ea-8188-738512c3251a None admin] RBAC: Authorizing `identity:create_grant(project_id=4882fc69f4c84744810e6cdfcbfc16df, user_id=f8ff084606084f0fb810529a2bea08e9, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:41.124972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cae458-a7c5-41ea-8188-738512c3251a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:41.146642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:41.159020 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 69/136] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:43:40 2022] PUT /identity/v3/projects/4882fc69f4c84744810e6cdfcbfc16df/users/f8ff084606084f0fb810529a2bea08e9/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:43:41.216670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e53ad-6bcf-4ba6-8009-2f4a7e7fec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:41.218571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e53ad-6bcf-4ba6-8009-2f4a7e7fec0c None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:41.219739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e53ad-6bcf-4ba6-8009-2f4a7e7fec0c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:41.220697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e53ad-6bcf-4ba6-8009-2f4a7e7fec0c None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/f8ff084606084f0fb810529a2bea08e9/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:41.229947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e53ad-6bcf-4ba6-8009-2f4a7e7fec0c None admin] RBAC: Authorizing `identity:create_grant(project_id=4010acd13d9445b1943eac27cb61aaa2, user_id=f8ff084606084f0fb810529a2bea08e9, role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:41.233625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e53ad-6bcf-4ba6-8009-2f4a7e7fec0c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:41.278591 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 68/137] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:43:40 2022] PUT /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/f8ff084606084f0fb810529a2bea08e9/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:43:41.600874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52788772-6cf2-450f-b067-71f6bee9c16e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:41.601363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52788772-6cf2-450f-b067-71f6bee9c16e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:41.601743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52788772-6cf2-450f-b067-71f6bee9c16e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:41.604937 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 70/138] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:41.613924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a725-8dc4-4e97-b402-576a61b1c0bd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:41.614910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a725-8dc4-4e97-b402-576a61b1c0bd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:41.615706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a725-8dc4-4e97-b402-576a61b1c0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:41.639981 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2b05-2d41-4740-8d76-6e5368ad3bac None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:41.640459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2b05-2d41-4740-8d76-6e5368ad3bac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:41.640878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2b05-2d41-4740-8d76-6e5368ad3bac None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:41.643063 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 71/139] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:43:41.657407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bbad-7718-439f-b9f9-84c6d638223b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:41.657909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bbad-7718-439f-b9f9-84c6d638223b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:41.658319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980bbad-7718-439f-b9f9-84c6d638223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:41.662014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-46b9a725-8dc4-4e97-b402-576a61b1c0bd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:41.664954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-46b9a725-8dc4-4e97-b402-576a61b1c0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:41.699273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:41.708413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9980bbad-7718-439f-b9f9-84c6d638223b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:41.714517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9980bbad-7718-439f-b9f9-84c6d638223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:41.878405 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 69/140] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:41.883933 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 72/141] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:42.557391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9a725-8dc4-4e97-b402-576a61b1c0bd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:42.791902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9a725-8dc4-4e97-b402-576a61b1c0bd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:42.865734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:42.866730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:42.867144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:42.867552 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:42.881652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980bbad-7718-439f-b9f9-84c6d638223b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:42.894515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:42.909159 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:42.910704 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:42.912072 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 70/142] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:42.923799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:42.952221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:43.332172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980bbad-7718-439f-b9f9-84c6d638223b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:43.373468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f72d-851f-4a32-a258-c71eb186e0a2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:43.446346 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:43.489594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:43.490640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:43.491101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:43.491574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:43.497833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:43.500740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:43.504455 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 73/143] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 624 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:43.514324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:43.514571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:43.515604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:43.516077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:43.516604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:43.517886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:43.518430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:43.521211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:43.556299 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 71/144] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 634 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:43.567319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:43.790784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:43.907269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f22fb-bacf-4ee5-9b2e-8cf91444f93e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:43.951781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4692837-bf7b-4484-b500-7fecbd9d1baa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:44.040494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:44.042260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:44.043085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:44.043811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] PATH_INFO: `/v3/users/c21c9bc05bc340358a9ecde27feaeeaf` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:44.053250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=c21c9bc05bc340358a9ecde27feaeeaf)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:44.056319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:44.062273 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 72/145] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:43 2022] GET /identity/v3/users/c21c9bc05bc340358a9ecde27feaeeaf => generated 289 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:43:44.072863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:44.114850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:44.116250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:44.117025 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:44.117757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:44.120960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:44.124080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:44.258600 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 74/146] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:43:43 2022] POST /identity/v3/users => generated 283 bytes in 745 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:43:44.404726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90636aa2-915b-468d-a65b-377d5a87a6d1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:44.482464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb2bb-710d-4b4c-8289-cfbe533f06f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:44.483594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb2bb-710d-4b4c-8289-cfbe533f06f5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:44.484058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb2bb-710d-4b4c-8289-cfbe533f06f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:44.484579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb2bb-710d-4b4c-8289-cfbe533f06f5 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:44.486925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb2bb-710d-4b4c-8289-cfbe533f06f5 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:44.489826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb2bb-710d-4b4c-8289-cfbe533f06f5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:44.493673 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 73/147] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:44 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:44.504562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:44.649066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:44.768832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5b088-50bc-410c-ba49-c7869069400f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:44.849841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9076c-8d82-4ec5-aa0d-7cf85d985fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:44.851004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9076c-8d82-4ec5-aa0d-7cf85d985fde None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:44.851496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9076c-8d82-4ec5-aa0d-7cf85d985fde None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:44.851975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9076c-8d82-4ec5-aa0d-7cf85d985fde None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/c21c9bc05bc340358a9ecde27feaeeaf/roles/fef78cf3dcc64b77a03f0b53167b7430` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:44.857961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9076c-8d82-4ec5-aa0d-7cf85d985fde None admin] RBAC: Authorizing `identity:create_grant(project_id=4010acd13d9445b1943eac27cb61aaa2, user_id=c21c9bc05bc340358a9ecde27feaeeaf, role_id=fef78cf3dcc64b77a03f0b53167b7430)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:44.860896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9076c-8d82-4ec5-aa0d-7cf85d985fde None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:44.890726 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 75/148] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:43:44 2022] PUT /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2/users/c21c9bc05bc340358a9ecde27feaeeaf/roles/fef78cf3dcc64b77a03f0b53167b7430 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:43:45.346687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913aac9-6391-49d0-8b80-5c4fdb8bc580 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:45.346687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913aac9-6391-49d0-8b80-5c4fdb8bc580 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:45.346687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913aac9-6391-49d0-8b80-5c4fdb8bc580 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:45.347656 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 74/149] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:45.357094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7aaa-960d-40f5-aa0d-2b0aa39af247 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:45.357749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7aaa-960d-40f5-aa0d-2b0aa39af247 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:45.358043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7aaa-960d-40f5-aa0d-2b0aa39af247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:45.400869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ee5b7aaa-960d-40f5-aa0d-2b0aa39af247 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:45.403081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ee5b7aaa-960d-40f5-aa0d-2b0aa39af247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:45.586051 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 76/150] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:46.319458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913aac9-6391-49d0-8b80-5c4fdb8bc580 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:46.327701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:46.568676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913aac9-6391-49d0-8b80-5c4fdb8bc580 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:46.579612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f170712-6807-441d-a13b-e16223c2b4bb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:46.581166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f170712-6807-441d-a13b-e16223c2b4bb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:46.581842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f170712-6807-441d-a13b-e16223c2b4bb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:46.586906 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 77/151] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:46.598338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082ee50-1fec-4a1b-ac4f-5760206e9898 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:46.599304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082ee50-1fec-4a1b-ac4f-5760206e9898 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:46.601004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082ee50-1fec-4a1b-ac4f-5760206e9898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:46.666064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f082ee50-1fec-4a1b-ac4f-5760206e9898 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:46.670164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f082ee50-1fec-4a1b-ac4f-5760206e9898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:46.687635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:46.688849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:46.689337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:46.689809 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:46.716860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:46.727711 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:46.729319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:46.730723 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 75/152] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:46.739359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:46.799184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:46.889588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:46.907358 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 78/153] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:46.983633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a751b1-5945-4476-ae9d-a1e8d4c2e31d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:47.052763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2a17-3185-48e0-8375-fed73a681ced None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:47.053286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2a17-3185-48e0-8375-fed73a681ced None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:47.053784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d2a17-3185-48e0-8375-fed73a681ced None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:47.056470 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 79/154] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:47.070521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836173a-5575-4782-ab31-46ea28ef0935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:47.071808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836173a-5575-4782-ab31-46ea28ef0935 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:47.072317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836173a-5575-4782-ab31-46ea28ef0935 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:47.072829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836173a-5575-4782-ab31-46ea28ef0935 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:47.074180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836173a-5575-4782-ab31-46ea28ef0935 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:47.074718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836173a-5575-4782-ab31-46ea28ef0935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:47.077534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836173a-5575-4782-ab31-46ea28ef0935 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:47.100767 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 76/155] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:47.111583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d2a17-3185-48e0-8375-fed73a681ced None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:47.114848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1bc7-c00f-4bf4-86fa-9a9d853ef31e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:47.115352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1bc7-c00f-4bf4-86fa-9a9d853ef31e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:47.115827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1bc7-c00f-4bf4-86fa-9a9d853ef31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:47.162752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-94ae1bc7-c00f-4bf4-86fa-9a9d853ef31e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:47.164956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-94ae1bc7-c00f-4bf4-86fa-9a9d853ef31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:47.358354 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 77/156] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:43:47.366560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f61c2-a057-4dc1-a23b-71c806e112e4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:47.367212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f61c2-a057-4dc1-a23b-71c806e112e4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:47.367656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f61c2-a057-4dc1-a23b-71c806e112e4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:47.370648 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 78/157] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:47.380245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0088fa2-412a-49f2-86a2-067c005d36e5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:47.381172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0088fa2-412a-49f2-86a2-067c005d36e5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:47.381808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0088fa2-412a-49f2-86a2-067c005d36e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:47.426212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f0088fa2-412a-49f2-86a2-067c005d36e5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:47.433802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f0088fa2-412a-49f2-86a2-067c005d36e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:47.561671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d2a17-3185-48e0-8375-fed73a681ced None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:47.589845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:47.629044 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 79/158] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:47.689879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:47.690860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:47.691359 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:47.691848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:47.694119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:47.700441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:47.703784 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 80/159] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:47 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 593 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:47.713348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0088fa2-412a-49f2-86a2-067c005d36e5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:47.952515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0088fa2-412a-49f2-86a2-067c005d36e5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:48.024923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:48.026018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:48.030139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:48.030706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:48.106714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:48.164531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:48.203872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:48.206553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:48.215004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:48.417043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29475b-c9ea-492a-8660-51d2c491f840 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:48.503265 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 80/160] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 791 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:48.508869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:48.539476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:48.540629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:48.541132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:48.541631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:48.574605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:48.601216 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:48.603123 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:43:48.604854 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 81/161] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:43:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 498 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:48.609619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:48.801402 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:48.859779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2fd6f-6959-432b-9556-db0531f56be8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:48.925068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:48.927515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ab3a0-8d1d-442f-9b27-b6837b0a6242 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:48.999284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:49.000422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:49.000913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:49.001397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:49.032024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:49.033207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:49.033690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:49.034175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:49.036446 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:49.039531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:49.043433 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 82/162] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:48 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:49.048053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:49.053503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:49.069450 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:49.071131 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:49.072677 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 81/163] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:43:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 568 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:49.076305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:49.496307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2547-47fc-45a2-8027-a9d2c7061367 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:49.500452 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626f29a-dbb3-4c8d-9308-218962d42731 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:49.575945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:49.577529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:49.578321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:49.579077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:49.602144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:49.623383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:49.625515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:43:49.627056 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:49.631867 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 82/164] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 560 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:49.633886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:49.660976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:49.713825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:49.714979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:49.715438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:49.715920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:49.717288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:49.717837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:49.720654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:49.760128 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 83/165] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 716 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:49.762596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:49.936407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd2b78-1a7b-4329-a9cf-a532ca3b9364 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:49.939540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:50.013815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:50.015832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:50.016327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:50.016844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:50.018297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=f8ff084606084f0fb810529a2bea08e9) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:50.018852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:50.021470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:50.034748 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 83/166] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:49 2022] GET /identity/v3/role_assignments?user.id=f8ff084606084f0fb810529a2bea08e9&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=896da438cb834959bd42187c944ea2f7 => generated 608 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:50.036004 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:50.249476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:50.257655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148948e-bc39-4e96-9774-d16ad3b49ffc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:50.358336 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafef39-c17f-4e5b-b79f-b3df22493f3e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:50.366136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:50.367315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:50.367890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:50.368373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:50.374020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:50.374525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:50.377248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:50.417850 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 84/167] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:43:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 657 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:50.419409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:50.532881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:50.533964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:50.534433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:50.534910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:50.536192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:50.536786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:50.546605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:50.583551 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 84/168] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 548 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:50.585133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:50.759723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:50.796436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6751d-b09c-4933-94ce-39f21bb45847 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:50.947664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:50.949008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:50.952300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:50.952300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] PATH_INFO: `/v3/users/7935e75bd19b46fd9203b785ff55ae84` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:50.952693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383643c-b6d4-4581-ac2b-5969b326757f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:50.957826 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] RBAC: Authorizing `identity:get_user(user_id=7935e75bd19b46fd9203b785ff55ae84)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:50.960682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:50.964490 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 85/169] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:50 2022] GET /identity/v3/users/7935e75bd19b46fd9203b785ff55ae84 => generated 247 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:50.966030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:51.037937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:51.066900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:51.067931 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:51.070381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:51.070381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:51.071833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:51.074943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:51.082845 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 85/170] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:50 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:51.084066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:51.320721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02075d-e13e-45a4-b1cd-0db2eb3d8a9a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:51.377451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:51.378231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:51.378522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:51.378806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] PATH_INFO: `/v3/users/f8ff084606084f0fb810529a2bea08e9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:51.386730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] RBAC: Authorizing `identity:get_user(user_id=f8ff084606084f0fb810529a2bea08e9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:51.389548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:51.392145 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 86/171] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:50 2022] GET /identity/v3/users/f8ff084606084f0fb810529a2bea08e9 => generated 275 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:51.393118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:51.469575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:51.492674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99285855-474b-4373-ba3a-9b78528c7df8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:51.599945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:51.601490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:51.602375 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:51.603125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:51.732257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017042f-d88e-41e8-9abb-87017e7a12c2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:51.844065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:51.851862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:51.855172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:51.858851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bc0b1-70f6-4936-9504-ae2470aca1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:51.859869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bc0b1-70f6-4936-9504-ae2470aca1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:51.860334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bc0b1-70f6-4936-9504-ae2470aca1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:51.860930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bc0b1-70f6-4936-9504-ae2470aca1d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:52.044628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bc0b1-70f6-4936-9504-ae2470aca1d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:52.046653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bc0b1-70f6-4936-9504-ae2470aca1d5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:52.087271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:52.197829 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 86/172] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1115 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:52.199451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:52.241034 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 87/173] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 848 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:52.249800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5947-3a74-48f8-bab0-9f9b9b22b784 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:52.250727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5947-3a74-48f8-bab0-9f9b9b22b784 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:52.251485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5947-3a74-48f8-bab0-9f9b9b22b784 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:52.253778 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 88/174] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:52.256045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5947-3a74-48f8-bab0-9f9b9b22b784 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:52.471802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5947-3a74-48f8-bab0-9f9b9b22b784 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:52.543454 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:52.555068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:52.562718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944ef28-ede2-4ec8-8d2b-e0ab106f3022 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:52.563891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80905e0-33db-4d33-ad84-a12b294edf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:52.565022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80905e0-33db-4d33-ad84-a12b294edf1a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:52.565508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80905e0-33db-4d33-ad84-a12b294edf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:52.565954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80905e0-33db-4d33-ad84-a12b294edf1a None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:52.568247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80905e0-33db-4d33-ad84-a12b294edf1a None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:52.571228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80905e0-33db-4d33-ad84-a12b294edf1a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:52.575790 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 89/175] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:52 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:52.578216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f27d-63ad-4fba-b020-2c642260fdf2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:52.578703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f27d-63ad-4fba-b020-2c642260fdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:52.579183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f27d-63ad-4fba-b020-2c642260fdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:52.615765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b446f27d-63ad-4fba-b020-2c642260fdf2 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:52.619123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b446f27d-63ad-4fba-b020-2c642260fdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:52.649556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763a9cf-cb92-4873-84ed-a9ad21d7bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:52.651472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763a9cf-cb92-4873-84ed-a9ad21d7bceb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:52.652300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763a9cf-cb92-4873-84ed-a9ad21d7bceb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:52.653368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763a9cf-cb92-4873-84ed-a9ad21d7bceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:52.769012 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 90/176] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:52.771407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446f27d-63ad-4fba-b020-2c642260fdf2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:52.797731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763a9cf-cb92-4873-84ed-a9ad21d7bceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:52.801167 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763a9cf-cb92-4873-84ed-a9ad21d7bceb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:52.946474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446f27d-63ad-4fba-b020-2c642260fdf2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:52.970007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:52.982422 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 87/177] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 783 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:52.982841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c957a-598d-459f-98a3-adc3308b7aca None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:52.983343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c957a-598d-459f-98a3-adc3308b7aca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:52.983832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c957a-598d-459f-98a3-adc3308b7aca None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:52.986503 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 88/178] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:52.992247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:52.994523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c957a-598d-459f-98a3-adc3308b7aca None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:53.007088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9104d43-f5ee-4915-b29e-cd2802be4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:53.007927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104d43-f5ee-4915-b29e-cd2802be4f14 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.008342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104d43-f5ee-4915-b29e-cd2802be4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.008792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9104d43-f5ee-4915-b29e-cd2802be4f14 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.010412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9104d43-f5ee-4915-b29e-cd2802be4f14 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:53.012756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9104d43-f5ee-4915-b29e-cd2802be4f14 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:53.020571 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 91/179] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:52 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:53.020571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed79e3-5df1-4c4e-b5a0-9fc7bc88df26 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.020571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed79e3-5df1-4c4e-b5a0-9fc7bc88df26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.020571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed79e3-5df1-4c4e-b5a0-9fc7bc88df26 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.021227 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 92/180] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:53.023652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfb873-939e-4a9b-8295-7a9aee777b42 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.024487 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfb873-939e-4a9b-8295-7a9aee777b42 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.025213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfb873-939e-4a9b-8295-7a9aee777b42 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.028012 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 93/181] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:53.029861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfb873-939e-4a9b-8295-7a9aee777b42 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:53.217782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c957a-598d-459f-98a3-adc3308b7aca None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:53.229274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfb873-939e-4a9b-8295-7a9aee777b42 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:53.280837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd7a82-3135-4783-8043-e77d2b4de557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:53.281430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd7a82-3135-4783-8043-e77d2b4de557 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.281821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd7a82-3135-4783-8043-e77d2b4de557 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.282187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd7a82-3135-4783-8043-e77d2b4de557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.283083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd7a82-3135-4783-8043-e77d2b4de557 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=7935e75bd19b46fd9203b785ff55ae84) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:53.283593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd7a82-3135-4783-8043-e77d2b4de557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:53.285435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd7a82-3135-4783-8043-e77d2b4de557 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:53.294755 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 89/182] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:52 2022] GET /identity/v3/role_assignments?user.id=7935e75bd19b46fd9203b785ff55ae84&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:53.296307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:53.296566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c481abb-11f3-45f3-9086-491212ea0ba5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.297015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c481abb-11f3-45f3-9086-491212ea0ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.297353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.297497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c481abb-11f3-45f3-9086-491212ea0ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.297801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.298231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] PATH_INFO: `/v3/projects/4882fc69f4c84744810e6cdfcbfc16df` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.300289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] RBAC: Authorizing `identity:get_project(project_id=4882fc69f4c84744810e6cdfcbfc16df)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:53.303005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:53.307070 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 94/183] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:53 2022] GET /identity/v3/projects/4882fc69f4c84744810e6cdfcbfc16df => generated 312 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:53.309756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:53.331563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5c481abb-11f3-45f3-9086-491212ea0ba5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:53.333175 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c481abb-11f3-45f3-9086-491212ea0ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:53.462392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:53.471521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:53.493269 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 90/184] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:53.495979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dac13-6061-43b4-8291-eda34f82f154 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.496679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dac13-6061-43b4-8291-eda34f82f154 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.497191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dac13-6061-43b4-8291-eda34f82f154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.535810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-545dac13-6061-43b4-8291-eda34f82f154 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:53.539729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-545dac13-6061-43b4-8291-eda34f82f154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:53.557267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6215bf-aed9-4146-8f1b-8823759f0a1a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:53.633781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687730dd-2836-4231-8899-0d8675e292f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:53.634782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687730dd-2836-4231-8899-0d8675e292f9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.635208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687730dd-2836-4231-8899-0d8675e292f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.636476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687730dd-2836-4231-8899-0d8675e292f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.637971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687730dd-2836-4231-8899-0d8675e292f9 None admin] RBAC: Adding query filter params (role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=f8ff084606084f0fb810529a2bea08e9) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:53.639105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687730dd-2836-4231-8899-0d8675e292f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:53.641441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687730dd-2836-4231-8899-0d8675e292f9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:53.658149 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 95/185] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:53 2022] GET /identity/v3/role_assignments?user.id=f8ff084606084f0fb810529a2bea08e9&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 608 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:53.658149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b38ef-e2d5-4100-a118-941d62d1f481 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:53.658149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b38ef-e2d5-4100-a118-941d62d1f481 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:53.658149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b38ef-e2d5-4100-a118-941d62d1f481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:53.690829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7b8b38ef-e2d5-4100-a118-941d62d1f481 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:53.693067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7b8b38ef-e2d5-4100-a118-941d62d1f481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:53.702180 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 91/186] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:53.703454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dac13-6061-43b4-8291-eda34f82f154 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:53.944791 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 96/187] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:53.945537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b38ef-e2d5-4100-a118-941d62d1f481 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:54.016681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dac13-6061-43b4-8291-eda34f82f154 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:54.066777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:54.091538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:54.092200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:54.092643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:54.092947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:54.103920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:54.126349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:54.132746 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:54.134501 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 92/188] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:54.137377 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:54.252331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b38ef-e2d5-4100-a118-941d62d1f481 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:54.316698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:54.317454 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:54.317725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:54.318286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:54.319207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=4882fc69f4c84744810e6cdfcbfc16df, user.id=f8ff084606084f0fb810529a2bea08e9) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:54.319518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:54.321297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:54.326017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96c55-6120-4c7a-9bad-47b175d152f3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:54.330257 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 97/189] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:53 2022] GET /identity/v3/role_assignments?user.id=f8ff084606084f0fb810529a2bea08e9&scope.project.id=4882fc69f4c84744810e6cdfcbfc16df&role.id=896da438cb834959bd42187c944ea2f7 => generated 608 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:54.340696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:54.401343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:54.402494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:54.402982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:54.403464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:54.425389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:54.435932 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:54.437076 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:43:54.437879 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:54.439125 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 93/190] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:43:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:54.440098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:54.548612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a52238-78b7-4b4b-a885-6b20ebd5538f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:54.561656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:54.594889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:54.632077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:54.632886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:54.633301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:54.633639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:54.634489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:54.634992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:54.638171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:54.650474 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 98/191] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:43:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:54.651316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:54.687206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bca28d-8c98-46cb-9c54-51d75cabc349 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:54.755752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:54.756898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:54.757399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:54.757932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:54.760114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:54.763908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:54.767144 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 94/192] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:43:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:54.769578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:54.871992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7953-da6f-44ed-adff-6ea2f26b539f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:54.937194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:54.938272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:54.938672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:54.938959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:54.940200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:54.940727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:54.943342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:54.954595 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 99/193] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:43:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:54.956559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:54.959983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f9719-2cde-4dcf-85a8-871b2649d8b2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:54.978919 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:55.019146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:55.020252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:55.020800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:55.021303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:55.028223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:55.041271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:43:55.051392 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:43:55.053080 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:43:55.054587 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 95/194] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:43:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:43:55.056957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:55.105347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985698e-2d66-427c-adf2-d6afd164d24a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:55.161162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:55.161817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:55.162258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:55.162676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] PATH_INFO: `/v3/users/c21c9bc05bc340358a9ecde27feaeeaf` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:55.164307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] RBAC: Authorizing `identity:get_user(user_id=c21c9bc05bc340358a9ecde27feaeeaf)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:55.166055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:55.169055 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 100/195] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:54 2022] GET /identity/v3/users/c21c9bc05bc340358a9ecde27feaeeaf => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:55.171670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:55.244098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54e15b-f3ae-4285-af6b-f330e7a29657 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:55.291169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:55.292335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:55.292805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:55.293178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:55.294118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:55.295980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:55.330508 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 96/196] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:43:55 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:43:55.331673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:55.382854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4065d-3cde-44c9-a731-a0507ca730fa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:55.391113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:55.412744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:55.454939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:55.456055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:55.456621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:55.457079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] PATH_INFO: `/v3/users/c8bf93809fd84c1d8fa93606d809cd95` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:55.484536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=c8bf93809fd84c1d8fa93606d809cd95)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:55.487475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:55.497028 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 101/197] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:55 2022] GET /identity/v3/users/c8bf93809fd84c1d8fa93606d809cd95 => generated 283 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:55.500608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:55.571848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb4a19-27c1-43bb-97af-459b1c35054b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:55.649473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:55.650889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:55.652126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:55.655305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:55.656746 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:55.659877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:55.660120 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:55.675883 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 97/198] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:43:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:55.677384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:55.720920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28269c-1ded-4111-98e6-af4657d4c6b4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:55.803822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:55.805295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:55.806035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:55.806729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:55.895552 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:55.902368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:55.918742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e97780-4fb3-4565-b041-99d609afad9e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:55.976216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:55.979606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:56.000532 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:56.002271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:56.003223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:56.004793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:56.178780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:56.182233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:56.220574 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 102/199] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:56.220574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:56.379185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:56.407213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:56.435456 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 98/200] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:56.437373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:56.460623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2e864-cbb6-41c1-874c-ddfd6472f5f6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:56.536938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:56.537836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:56.538121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:56.538401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] PATH_INFO: `/v3/users/c8bf93809fd84c1d8fa93606d809cd95` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:56.541037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] RBAC: Authorizing `identity:get_user(user_id=c8bf93809fd84c1d8fa93606d809cd95)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:56.544111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:56.547947 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 103/201] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:43:56 2022] GET /identity/v3/users/c8bf93809fd84c1d8fa93606d809cd95 => generated 283 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:56.549178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:56.665232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e2e1f-2c69-405b-b8d1-4dcbd2f1bf63 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:56.737149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:56.738274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:56.738895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:56.739578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:56.742103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:56.744286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:56.746571 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 99/202] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:56 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:56.748290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:56.775874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8029770-5d9f-46f1-b8a9-db512f98a2c6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:56.860936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:56.862029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:56.862523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:56.863008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:56.865368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:56.869530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:56.873815 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 104/203] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:56 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:56.878179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:56.887874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:56.901925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:56.999644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7de9e-9d80-4c52-9eeb-19f28187eeaa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:57.085654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:57.087195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:57.088055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:57.089964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:57.148181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175e21a-2914-406f-9ed3-f4897306e51f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:57.225351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:57.226569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:57.227048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:57.227460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:57.228290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] RBAC: Adding query filter params (role.id=fef78cf3dcc64b77a03f0b53167b7430, scope.project.id=4010acd13d9445b1943eac27cb61aaa2, user.id=c21c9bc05bc340358a9ecde27feaeeaf) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:57.228963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:57.232153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:57.245584 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 105/204] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:56 2022] GET /identity/v3/role_assignments?user.id=c21c9bc05bc340358a9ecde27feaeeaf&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=fef78cf3dcc64b77a03f0b53167b7430 => generated 608 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:57.247749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:57.259782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:57.263476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:57.392112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:57.396169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:57.460937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c6db2-3e48-45b0-8160-fa11b8dd43f1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:57.532135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:57.533431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:57.534009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:57.534574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:57.536274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] RBAC: Adding query filter params (role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=c8bf93809fd84c1d8fa93606d809cd95) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:57.536961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:57.540260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:57.552031 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 106/205] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:57 2022] GET /identity/v3/role_assignments?user.id=c8bf93809fd84c1d8fa93606d809cd95&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:57.593618 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 100/206] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:43:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 846 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:43:57.602515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:57.839803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c34c-fac1-4a20-977d-9b1af0398e2b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:57.916832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12e0e9-2d36-4bcb-8943-1f2e1541118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:57.918012 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12e0e9-2d36-4bcb-8943-1f2e1541118a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:57.918479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12e0e9-2d36-4bcb-8943-1f2e1541118a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:57.918933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12e0e9-2d36-4bcb-8943-1f2e1541118a None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:57.921326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12e0e9-2d36-4bcb-8943-1f2e1541118a None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:57.924232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12e0e9-2d36-4bcb-8943-1f2e1541118a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:57.927429 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 107/207] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:43:57 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:57.938019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:43:58.171941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05d07d-3910-41a1-87da-39d09c967d38 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:43:58.240667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4f0d8-99cb-4030-81fb-c06dc87f94dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:43:58.241997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f0d8-99cb-4030-81fb-c06dc87f94dc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:58.242693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f0d8-99cb-4030-81fb-c06dc87f94dc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:58.243393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f0d8-99cb-4030-81fb-c06dc87f94dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:58.245062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4f0d8-99cb-4030-81fb-c06dc87f94dc None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=c8bf93809fd84c1d8fa93606d809cd95) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:43:58.245864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4f0d8-99cb-4030-81fb-c06dc87f94dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:43:58.248853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4f0d8-99cb-4030-81fb-c06dc87f94dc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:43:58.259908 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 101/208] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:43:57 2022] GET /identity/v3/role_assignments?user.id=c8bf93809fd84c1d8fa93606d809cd95&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:43:59.653768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667081f-d838-4724-a0d6-3d8a3c0be409 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:59.654541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667081f-d838-4724-a0d6-3d8a3c0be409 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:59.654963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667081f-d838-4724-a0d6-3d8a3c0be409 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:59.657675 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 108/209] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:43:59.668022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c640b6-eec9-4c66-921b-65e5745535e9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:59.668824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c640b6-eec9-4c66-921b-65e5745535e9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:59.669316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c640b6-eec9-4c66-921b-65e5745535e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:59.702264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e3c640b6-eec9-4c66-921b-65e5745535e9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:59.704449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e3c640b6-eec9-4c66-921b-65e5745535e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:59.741300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:59.744154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79406fc-1554-4c71-9102-7ee31261e1bf None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:59.745135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79406fc-1554-4c71-9102-7ee31261e1bf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:59.745404 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79406fc-1554-4c71-9102-7ee31261e1bf None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:59.748094 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 109/210] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:43:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:43:59.756813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb678ca-3a44-4009-8287-9ebdba50911a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:43:59.757236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb678ca-3a44-4009-8287-9ebdba50911a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:43:59.757676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb678ca-3a44-4009-8287-9ebdba50911a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:43:59.768339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:43:59.793742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cfb678ca-3a44-4009-8287-9ebdba50911a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:43:59.796234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cfb678ca-3a44-4009-8287-9ebdba50911a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:43:59.861105 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 102/211] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:43:59.949392 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 110/212] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:00.379016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c640b6-eec9-4c66-921b-65e5745535e9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:00.474661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb678ca-3a44-4009-8287-9ebdba50911a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:00.600961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c640b6-eec9-4c66-921b-65e5745535e9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:00.671314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:00.673543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:00.674241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:00.674941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:00.688056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb678ca-3a44-4009-8287-9ebdba50911a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:00.696366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:00.706318 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:00.707915 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:00.709458 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:00.714571 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 103/213] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:00.731286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:00.753590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:00.762330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:00.766296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:00.767389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:00.767880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:00.768498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:00.770775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:00.773583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:00.777173 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 111/214] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:44:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:00.787068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:00.941588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce406edc-e48f-4527-b597-ec9e2e1991a0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:00.996925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880c7f-9891-43aa-91c5-ceba64fe4563 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:01.007732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:01.008754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:01.009224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:01.009695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:01.011710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:01.012227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:01.014886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:01.031981 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 104/215] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:01.042226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:01.066242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:01.067691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:01.068406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:01.068884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:01.070119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:01.084425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:01.117089 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 112/216] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:44:00 2022] POST /identity/v3/users => generated 297 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:44:01.208901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:01.264257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce6c8-4f23-4f39-8938-4954852dbedb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:01.342036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d774e25-0154-461f-95e2-8ebef4ca0372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:01.351282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d774e25-0154-461f-95e2-8ebef4ca0372 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:01.351282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d774e25-0154-461f-95e2-8ebef4ca0372 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:01.351282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d774e25-0154-461f-95e2-8ebef4ca0372 None admin] PATH_INFO: `/v3/users/139c6db27de04e098a1b7637397cacd7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:01.389106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d774e25-0154-461f-95e2-8ebef4ca0372 None admin] RBAC: Authorizing `identity:get_user(user_id=139c6db27de04e098a1b7637397cacd7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:01.391982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d774e25-0154-461f-95e2-8ebef4ca0372 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:01.407602 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 105/217] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:01 2022] GET /identity/v3/users/139c6db27de04e098a1b7637397cacd7 => generated 297 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:01.422871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:01.537758 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:01.582838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6009e-19a6-411f-a693-e3d93e446515 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:01.583135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6009e-19a6-411f-a693-e3d93e446515 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:01.584488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6009e-19a6-411f-a693-e3d93e446515 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:01.585897 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 106/218] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:44:01.596578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9982b-4c5f-4f8c-906d-e13962e4e3d5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:01.597024 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9982b-4c5f-4f8c-906d-e13962e4e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:01.597805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9982b-4c5f-4f8c-906d-e13962e4e3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:01.636203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f2a9982b-4c5f-4f8c-906d-e13962e4e3d5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:01.639261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f2a9982b-4c5f-4f8c-906d-e13962e4e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:01.658199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bac844-c27d-4161-b32f-59ed7cb9c082 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:01.751587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:01.755123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:01.756011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:01.757143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:01.804451 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 107/219] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:01.947448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:01.950302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:02.072075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:02.206711 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 113/220] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 785 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 25 00:44:02.222273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:02.222988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7a0d-0cdb-47c3-aece-01b5a90cad5e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:02.223535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7a0d-0cdb-47c3-aece-01b5a90cad5e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:02.224077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7a0d-0cdb-47c3-aece-01b5a90cad5e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:02.240418 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 108/221] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:02 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:02.252433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62229e0-dbc7-4fa0-9d4a-9fd4e1521af1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:02.252433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62229e0-dbc7-4fa0-9d4a-9fd4e1521af1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:02.252433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62229e0-dbc7-4fa0-9d4a-9fd4e1521af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:02.359539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b62229e0-dbc7-4fa0-9d4a-9fd4e1521af1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:02.370544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b62229e0-dbc7-4fa0-9d4a-9fd4e1521af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:02.490320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5c05-8ca8-47ee-a482-fa82a5d284e4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:02.568193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:02.592144 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:02.593286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:02.593830 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 109/222] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:02.600420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:02.600420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:02.600420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:02.600420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:02.602340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62229e0-dbc7-4fa0-9d4a-9fd4e1521af1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:02.610783 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 114/223] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:02 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:02.612272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:02.698463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:02.866357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725441ba-661f-42b3-8b6d-5b14793784f3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:02.947637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:02.949383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:02.950183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:02.950977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:02.954045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=139c6db27de04e098a1b7637397cacd7) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:02.955290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:02.960166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:02.967259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62229e0-dbc7-4fa0-9d4a-9fd4e1521af1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:02.973315 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 115/224] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:44:02 2022] GET /identity/v3/role_assignments?user.id=139c6db27de04e098a1b7637397cacd7&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=896da438cb834959bd42187c944ea2f7 => generated 259 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:03.095815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:03.096851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:03.097277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:03.097693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:03.122273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:03.137665 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:03.139346 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:03.140816 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 110/225] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 545 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:03.149543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:03.248504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:03.386760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:03.413728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d4a0f-e06f-4b1f-a6fe-d195cd07a59a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:03.440635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:03.489262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:03.490341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:03.490852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:03.491345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:03.493162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:03.493162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:03.496611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:03.518251 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 116/226] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:03.525458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:03.572258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f5fe2-23af-4665-9e7c-0c342083ffc8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:03.717719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:03.719305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:03.720083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:03.720867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:03.741514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22ccb5-da74-4817-a5b1-c46c19bfb530 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:03.748253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:03.763028 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:03.764312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:03.765203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:03.767624 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 111/227] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:44:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 520 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:44:03.778597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:03.856558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:03.857627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:03.858102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:03.858588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] PATH_INFO: `/v3/users/c8bf93809fd84c1d8fa93606d809cd95` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:03.865981 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] RBAC: Authorizing `identity:get_user(user_id=c8bf93809fd84c1d8fa93606d809cd95)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:03.868351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:03.876532 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 117/228] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:03 2022] GET /identity/v3/users/c8bf93809fd84c1d8fa93606d809cd95 => generated 283 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:03.890432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:03.971735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac4498-64b6-46e1-982e-afcdcdec0beb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:03.972767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:03.999261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:04.053468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:04.054540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:04.055034 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:04.055483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:04.056794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:04.057433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:04.060172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:04.096010 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 112/229] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:04.113384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:04.144251 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879722e-d997-4529-a8d9-3d964f7a6347 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:04.221316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:04.222383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:04.222771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:04.223836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:04.224559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:04.226558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:04.228915 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 118/230] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:03 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:04.236634 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:04.338314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427661e-1c12-47c5-9ab9-cea8644026cb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:04.390968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:04.407170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:04.408769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:04.409606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:04.410359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] PATH_INFO: `/v3/users/c8bf93809fd84c1d8fa93606d809cd95` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:04.412850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] RBAC: Authorizing `identity:get_user(user_id=c8bf93809fd84c1d8fa93606d809cd95)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:04.415653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:04.416205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f4c7-717c-4071-a718-387d7abec445 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:04.419082 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 113/231] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:04 2022] GET /identity/v3/users/c8bf93809fd84c1d8fa93606d809cd95 => generated 283 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:04.427478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:04.514475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:04.515086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:04.515473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:04.516449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/c8bf93809fd84c1d8fa93606d809cd95/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:04.519159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:04.523167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] RBAC: Authorizing `identity:create_grant(project_id=a83742d0de534530bf5beab6ecf7fe70, user_id=c8bf93809fd84c1d8fa93606d809cd95, role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:04.525006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:04.561234 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 119/232] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:44:04 2022] PUT /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/c8bf93809fd84c1d8fa93606d809cd95/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:44:04.619840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7c0cc-83d3-4269-b9e5-3a91979396d2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:04.668893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2b7b2-6868-45e1-b562-51de65e960ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:04.669941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b7b2-6868-45e1-b562-51de65e960ae None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:04.670481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b7b2-6868-45e1-b562-51de65e960ae None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:04.670964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b7b2-6868-45e1-b562-51de65e960ae None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:04.672687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2b7b2-6868-45e1-b562-51de65e960ae None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:04.674501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2b7b2-6868-45e1-b562-51de65e960ae None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:04.676964 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 114/233] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:04 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:04.686240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:04.915313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816be8-a29e-4936-9757-0bc00fa0ab52 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:04.979836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2fe29-84ac-407f-b3b4-a714cc2573d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:04.981132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2fe29-84ac-407f-b3b4-a714cc2573d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:04.981639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2fe29-84ac-407f-b3b4-a714cc2573d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:04.982139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2fe29-84ac-407f-b3b4-a714cc2573d6 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/c8bf93809fd84c1d8fa93606d809cd95/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:04.985672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2fe29-84ac-407f-b3b4-a714cc2573d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a83742d0de534530bf5beab6ecf7fe70, user_id=c8bf93809fd84c1d8fa93606d809cd95, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:04.988668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2fe29-84ac-407f-b3b4-a714cc2573d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:05.024740 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 120/234] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:44:04 2022] PUT /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/c8bf93809fd84c1d8fa93606d809cd95/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:44:05.962585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5214af0-b765-4a38-b685-9aa4b479f979 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:05.963883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5214af0-b765-4a38-b685-9aa4b479f979 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:05.964266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5214af0-b765-4a38-b685-9aa4b479f979 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:05.966889 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 115/235] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:05.976216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8f8a0-0507-4677-a22d-da5dc46c74ba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:05.977060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8f8a0-0507-4677-a22d-da5dc46c74ba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:05.977203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8f8a0-0507-4677-a22d-da5dc46c74ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:06.006219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:06.018310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1de8f8a0-0507-4677-a22d-da5dc46c74ba None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:06.027510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1de8f8a0-0507-4677-a22d-da5dc46c74ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:06.175367 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 121/236] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:06.183061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6163d9-3456-4284-ba18-5ad41215c9a5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:06.183567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6163d9-3456-4284-ba18-5ad41215c9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:06.184068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6163d9-3456-4284-ba18-5ad41215c9a5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:06.186787 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 116/237] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:06.196544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bb097-e77b-4d39-9ebd-a08067ed9e0b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:06.197066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bb097-e77b-4d39-9ebd-a08067ed9e0b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:06.197574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bb097-e77b-4d39-9ebd-a08067ed9e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:06.238078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0a2bb097-e77b-4d39-9ebd-a08067ed9e0b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:06.240052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0a2bb097-e77b-4d39-9ebd-a08067ed9e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:06.387977 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 122/238] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:06.896139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6163d9-3456-4284-ba18-5ad41215c9a5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:07.110870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6163d9-3456-4284-ba18-5ad41215c9a5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:07.133482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:07.179599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:07.180549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:07.180969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:07.181374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:07.204559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bb097-e77b-4d39-9ebd-a08067ed9e0b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:07.205347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:07.213938 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:07.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:07.218051 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 117/239] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:07.226894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:07.361019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:07.430798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bb097-e77b-4d39-9ebd-a08067ed9e0b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:07.505170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:07.506464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:07.506945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:07.507431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:07.509744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:07.514003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:07.517714 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 123/240] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:44:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:07.527994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:07.623376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858753-8b9e-4f0e-ad9c-6cdd58e3f0c8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:07.697377 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c366ee-a870-4996-90b6-3bb5fb22ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:07.698799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c366ee-a870-4996-90b6-3bb5fb22ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:07.699254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c366ee-a870-4996-90b6-3bb5fb22ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:07.699703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c366ee-a870-4996-90b6-3bb5fb22ecd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:07.700878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c366ee-a870-4996-90b6-3bb5fb22ecd6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:07.701458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c366ee-a870-4996-90b6-3bb5fb22ecd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:07.703918 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c366ee-a870-4996-90b6-3bb5fb22ecd6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:07.724950 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 118/241] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:07.727262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41668957-3c04-4312-b6d6-56a668e18487 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:07.727709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41668957-3c04-4312-b6d6-56a668e18487 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:07.728174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41668957-3c04-4312-b6d6-56a668e18487 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:07.734150 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 119/242] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:07.740259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41668957-3c04-4312-b6d6-56a668e18487 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:07.752236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e48e3-e89e-41c9-aaec-40bd45363472 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:07.801931 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:07.823427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bae7e5-c9cf-4f42-8255-6ad44425ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:07.824580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bae7e5-c9cf-4f42-8255-6ad44425ca4b None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:07.825035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bae7e5-c9cf-4f42-8255-6ad44425ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:07.825484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bae7e5-c9cf-4f42-8255-6ad44425ca4b None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:07.826609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bae7e5-c9cf-4f42-8255-6ad44425ca4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:07.829428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bae7e5-c9cf-4f42-8255-6ad44425ca4b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:07.842818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:07.873159 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 124/243] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:44:07 2022] POST /identity/v3/users => generated 293 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:44:07.875876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d232b5-1a87-42a3-aa58-4f670ad51ae3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:07.876715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d232b5-1a87-42a3-aa58-4f670ad51ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:07.877363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d232b5-1a87-42a3-aa58-4f670ad51ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:07.909388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-28d232b5-1a87-42a3-aa58-4f670ad51ae3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:07.911683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-28d232b5-1a87-42a3-aa58-4f670ad51ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:07.982384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41668957-3c04-4312-b6d6-56a668e18487 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:08.043890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:08.045598 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:08.045598 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:08.045896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] PATH_INFO: `/v3/users/de701bb6edb34e0787c75b2445a69e24` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:08.070654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] RBAC: Authorizing `identity:get_user(user_id=de701bb6edb34e0787c75b2445a69e24)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:08.073544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:08.076675 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 120/244] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:07 2022] GET /identity/v3/users/de701bb6edb34e0787c75b2445a69e24 => generated 297 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:08.085880 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 125/245] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:08.086128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:08.196999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:08.209565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0423e-573f-4323-97b0-4e7d10e0b2e5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:08.256021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:08.256660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:08.256946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:08.257223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:08.394400 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:08.397433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:08.543609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:08.559635 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 121/246] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:08.566622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4d42-9b72-48a0-9b00-a91a97e8212d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:08.567338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:08.568219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4d42-9b72-48a0-9b00-a91a97e8212d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:08.571897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4d42-9b72-48a0-9b00-a91a97e8212d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:08.575980 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 126/247] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:08.585129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df9002-931d-48a9-ae67-06b9c18ee2fd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:08.585972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df9002-931d-48a9-ae67-06b9c18ee2fd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:08.586736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df9002-931d-48a9-ae67-06b9c18ee2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:08.631257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-70df9002-931d-48a9-ae67-06b9c18ee2fd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:08.634040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-70df9002-931d-48a9-ae67-06b9c18ee2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:08.702304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d806-c615-402e-803a-fdc02014445f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:08.771428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:08.772819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:08.773162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:08.773415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:08.775211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:08.777538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:08.782020 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 122/248] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:08 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:08.783853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:08.809620 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 127/249] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:08.811663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df9002-931d-48a9-ae67-06b9c18ee2fd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:08.913781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:08.942437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:09.032427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df9002-931d-48a9-ae67-06b9c18ee2fd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:09.055079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3bb5b-1581-46a3-ba4e-72b9753b889f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:09.100568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:09.101452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:09.101930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:09.102393 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:09.103720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] RBAC: Adding query filter params (role.id=fef78cf3dcc64b77a03f0b53167b7430, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=de701bb6edb34e0787c75b2445a69e24) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:09.105267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:09.107516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:09.118447 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 128/250] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:44:08 2022] GET /identity/v3/role_assignments?user.id=de701bb6edb34e0787c75b2445a69e24&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=fef78cf3dcc64b77a03f0b53167b7430 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:09.144430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:09.145898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:09.146608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:09.147259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:09.178620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:09.193181 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:09.195172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:09.197047 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:09.202095 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 123/251] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:09.211547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:09.521495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555f88c-acbd-448e-87e8-312e7e275c0d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:09.566476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:09.619235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:09.620964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:09.622192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:09.622192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:09.623104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:09.623633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:09.626838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:09.644214 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 129/252] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:09.654134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:09.870113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d73e01-37c7-4f49-8e2f-57343dcbba58 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:09.905044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:09.912441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:09.978186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:09.979901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:09.980426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:09.980922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] PATH_INFO: `/v3/users/139c6db27de04e098a1b7637397cacd7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:09.992610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] RBAC: Authorizing `identity:get_user(user_id=139c6db27de04e098a1b7637397cacd7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:09.995691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:10.002818 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 124/253] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:09 2022] GET /identity/v3/users/139c6db27de04e098a1b7637397cacd7 => generated 297 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:10.032468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:10.227772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905d38a-ec3a-4c72-848e-31d6e7c082c1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:10.250366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79ac7f-aafa-40ab-b171-af7c831a6749 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:10.305254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b4809-fd04-475c-bda3-34d95830984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:10.306780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b4809-fd04-475c-bda3-34d95830984b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:10.308046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b4809-fd04-475c-bda3-34d95830984b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:10.308806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b4809-fd04-475c-bda3-34d95830984b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:10.331042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb090f-e1f6-41c5-8c62-629b56037161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:10.336658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb090f-e1f6-41c5-8c62-629b56037161 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:10.338147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb090f-e1f6-41c5-8c62-629b56037161 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:10.340048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b4809-fd04-475c-bda3-34d95830984b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:10.341018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb090f-e1f6-41c5-8c62-629b56037161 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0f0b4809-fd04-475c-bda3-34d95830984b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:10.351934 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:10.353735 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:10.355902 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:10.359937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb090f-e1f6-41c5-8c62-629b56037161 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:10.363068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb090f-e1f6-41c5-8c62-629b56037161 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:10.366655 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 125/254] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:10 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:10.369230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ffc2-2244-4247-9992-4004ac49b51a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:10.371939 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 130/255] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:10.373013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ffc2-2244-4247-9992-4004ac49b51a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:10.373743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49ffc2-2244-4247-9992-4004ac49b51a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:10.376454 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 126/256] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:10.386023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e67873-0934-4776-83e0-631dc45d4a73 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:10.387971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ffc2-2244-4247-9992-4004ac49b51a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:10.388944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e67873-0934-4776-83e0-631dc45d4a73 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:10.389629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e67873-0934-4776-83e0-631dc45d4a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:10.449421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e5e67873-0934-4776-83e0-631dc45d4a73 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:10.453667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e5e67873-0934-4776-83e0-631dc45d4a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:10.586073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:10.606232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:10.716329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49ffc2-2244-4247-9992-4004ac49b51a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:10.751158 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 131/257] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:10.758869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e67873-0934-4776-83e0-631dc45d4a73 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:10.867876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:10.868902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:10.869383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:10.869915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:10.871105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:10.871589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:10.874166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:10.903346 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 127/258] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:10.920653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:11.160166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e67873-0934-4776-83e0-631dc45d4a73 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:11.269311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36db2c-320e-4e5e-b98d-168b310e7ab1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:11.301532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:11.302246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:11.302693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:11.303186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/139c6db27de04e098a1b7637397cacd7/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:11.306918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] RBAC: Authorizing `identity:create_grant(project_id=a83742d0de534530bf5beab6ecf7fe70, user_id=139c6db27de04e098a1b7637397cacd7, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:11.318181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:11.353078 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:11.374262 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 132/259] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:44:10 2022] PUT /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/139c6db27de04e098a1b7637397cacd7/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:44:11.419353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:11.420419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:11.420968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:11.421456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] PATH_INFO: `/v3/users/c8bf93809fd84c1d8fa93606d809cd95` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:11.426378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] RBAC: Authorizing `identity:get_user(user_id=c8bf93809fd84c1d8fa93606d809cd95)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:11.429190 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:11.432752 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 128/260] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:10 2022] GET /identity/v3/users/c8bf93809fd84c1d8fa93606d809cd95 => generated 283 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:11.441351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:11.513405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:11.522703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:11.781428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf2fcc-7f44-4d5f-b5b7-05a8e53a30a9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:11.832174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfead5-39c2-4c7d-9556-4311f5b7e38a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:11.899425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:11.900531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:11.901793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:11.902484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:11.939662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:11.951827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:11.953896 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:11.955901 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:11.960018 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 129/261] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:44:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:11.969806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:11.970927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:11.971431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:11.971928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:11.987476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:12.159204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:12.182343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:12.185379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:12.275306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:12.426062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67841ba-a753-47bf-998e-e50f3a727eb3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:12.537677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:12.538767 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:12.539236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:12.539709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:12.540986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:12.541606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:12.544261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:12.576424 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 130/262] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 590 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:12.592043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:12.593454 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 133/263] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1153 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:12.603008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:12.883230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37335b7e-1a62-41c3-9dbf-9aafd5b2bac5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:12.900230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:12.970872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:12.972443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:12.973133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:12.973983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] PATH_INFO: `/v3/users/c8bf93809fd84c1d8fa93606d809cd95` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:12.976249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c32f0-489d-4867-be6d-5c177a8f18b4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:12.977227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] RBAC: Authorizing `identity:get_user(user_id=c8bf93809fd84c1d8fa93606d809cd95)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:12.981123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:12.985100 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 131/264] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:12 2022] GET /identity/v3/users/c8bf93809fd84c1d8fa93606d809cd95 => generated 283 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:12.995299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:13.078655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:13.134674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:13.136460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:13.136966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:13.137566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:13.145852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:13.148713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:13.152222 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 134/265] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:12 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:13.162572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:13.353659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66146336-8154-4c7d-acd6-f1aeff007a0c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:13.518896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:13.520112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:13.520420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:13.520872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:13.633581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d1774-d884-4250-ac25-56d77c6925ec None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:13.708968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:13.760639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999c2f4-c8de-40b9-bfa7-a6e26d1343ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:13.760639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c2f4-c8de-40b9-bfa7-a6e26d1343ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:13.760639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c2f4-c8de-40b9-bfa7-a6e26d1343ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:13.760639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c2f4-c8de-40b9-bfa7-a6e26d1343ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:13.762219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999c2f4-c8de-40b9-bfa7-a6e26d1343ff None admin] RBAC: Adding query filter params (role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=c8bf93809fd84c1d8fa93606d809cd95) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:13.763648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999c2f4-c8de-40b9-bfa7-a6e26d1343ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:13.766761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999c2f4-c8de-40b9-bfa7-a6e26d1343ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:13.786953 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 135/266] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:44:13 2022] GET /identity/v3/role_assignments?user.id=c8bf93809fd84c1d8fa93606d809cd95&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 608 bytes in 625 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:13.820463 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:13.823976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:13.847765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c73ff-fb66-4288-9c62-c0c3c39e7572 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:13.848853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c73ff-fb66-4288-9c62-c0c3c39e7572 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:13.849617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c73ff-fb66-4288-9c62-c0c3c39e7572 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:13.852980 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 136/267] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:13.862691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69cf8-1229-4635-a74a-2275f28a5da4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:13.863602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69cf8-1229-4635-a74a-2275f28a5da4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:13.864337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69cf8-1229-4635-a74a-2275f28a5da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:13.919054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3bf69cf8-1229-4635-a74a-2275f28a5da4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:13.921592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3bf69cf8-1229-4635-a74a-2275f28a5da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:14.135171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:14.177345 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 132/268] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1183 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 25 00:44:14.188193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:14.191803 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 137/269] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:14.214795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5924-5a42-40ef-a27e-a413f1140134 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:14.215604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5924-5a42-40ef-a27e-a413f1140134 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:14.216290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5924-5a42-40ef-a27e-a413f1140134 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:14.219171 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 138/270] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:44:14.229804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437237f-a6b8-44b5-9688-9334c93aeb1b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:14.230664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437237f-a6b8-44b5-9688-9334c93aeb1b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:14.231359 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437237f-a6b8-44b5-9688-9334c93aeb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:14.286579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6437237f-a6b8-44b5-9688-9334c93aeb1b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:14.290572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6437237f-a6b8-44b5-9688-9334c93aeb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:14.435407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:14.463624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275706ce-dae6-409a-86f9-19a40a1bfaad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:14.529004 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 139/271] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:14.568741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adc161-1bdc-42c3-91c5-509bd6de3901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:14.570712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc161-1bdc-42c3-91c5-509bd6de3901 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:14.571384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc161-1bdc-42c3-91c5-509bd6de3901 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:14.572041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc161-1bdc-42c3-91c5-509bd6de3901 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:14.574972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adc161-1bdc-42c3-91c5-509bd6de3901 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:14.578047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adc161-1bdc-42c3-91c5-509bd6de3901 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:14.581573 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 133/272] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:14 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:14.591651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437237f-a6b8-44b5-9688-9334c93aeb1b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:14.714588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011100c9-dbdb-4966-8b7e-444bfaa68fa2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:14.715556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011100c9-dbdb-4966-8b7e-444bfaa68fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:14.716328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011100c9-dbdb-4966-8b7e-444bfaa68fa2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:14.727351 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 134/273] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:44:14.736119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436a9c-a81e-471b-b3a1-719718c4348f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:14.737069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436a9c-a81e-471b-b3a1-719718c4348f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:14.737895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436a9c-a81e-471b-b3a1-719718c4348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:14.802014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-06436a9c-a81e-471b-b3a1-719718c4348f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:14.804682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-06436a9c-a81e-471b-b3a1-719718c4348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:14.815192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437237f-a6b8-44b5-9688-9334c93aeb1b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:14.880952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:14.892026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:14.893749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:14.894499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:14.895158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:14.897805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=c8bf93809fd84c1d8fa93606d809cd95) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:14.898959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:14.903522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:14.916945 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 140/274] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:44:14 2022] GET /identity/v3/role_assignments?user.id=c8bf93809fd84c1d8fa93606d809cd95&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:14.919462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:15.006767 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 135/275] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 25 00:44:15.136931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1434eda-ad05-415b-8e6f-ffb685155327 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:15.150114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436a9c-a81e-471b-b3a1-719718c4348f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:15.214392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:15.215860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:15.216587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:15.218088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:15.241863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:15.258815 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:15.261412 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:15.263275 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:15.268249 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 141/276] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:15.280292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:15.377920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:15.419450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:15.479455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436a9c-a81e-471b-b3a1-719718c4348f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:15.568719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:15.570516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:15.573397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:15.573561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4e035-6207-4ce5-8e3b-c9b27863e4ac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:15.574359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:15.578584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:15.581882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:15.585754 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 136/277] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:44:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:15.606924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:15.656314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:15.657421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:15.657904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:15.658386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:15.659618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:15.660166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:15.662928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:15.680661 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 142/278] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:15.682679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:15.831620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc49745-ef71-42b1-a941-2f1995be7573 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:15.903655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:15.904658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:15.905238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:15.905699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:15.906880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:15.909536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:15.927586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c939f-4e2d-4496-8605-1b1987768c11 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:15.930340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:15.934694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:15.959561 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 137/279] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:44:15 2022] POST /identity/v3/users => generated 293 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:44:15.961429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:16.025246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:16.026512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:16.026978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:16.027505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:16.061419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:16.080044 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:16.081981 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:16.083563 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:16.090998 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 143/280] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:16.098509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:16.205677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747cc95-d9d5-4a37-a328-8ae70c03bbae None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:16.278828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:16.280002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:16.280561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:16.281043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] PATH_INFO: `/v3/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:16.309925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] RBAC: Authorizing `identity:get_user(user_id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:16.313307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:16.316041 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 138/281] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:15 2022] GET /identity/v3/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0 => generated 293 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:16.328198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:16.333474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a518ff0-f0a9-43a5-9122-e7b222f59348 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:16.413832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:16.414826 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:16.415273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:16.416020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:16.417615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:16.418115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:16.422397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:16.443311 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 144/282] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:16.455691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:16.468146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:16.527225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:16.582968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9ae6f-a133-4462-8b35-096c63c7be1e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:16.676535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f1a2f-d28e-41ef-9a02-b987abc25ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:16.677260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f1a2f-d28e-41ef-9a02-b987abc25ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:16.677760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f1a2f-d28e-41ef-9a02-b987abc25ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:16.678249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f1a2f-d28e-41ef-9a02-b987abc25ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:16.679670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f1a2f-d28e-41ef-9a02-b987abc25ce9 None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, user.id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0, scope.system=all) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:16.680231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f1a2f-d28e-41ef-9a02-b987abc25ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:16.683259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f1a2f-d28e-41ef-9a02-b987abc25ce9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:16.694098 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 139/283] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Sep 25 00:44:16 2022] GET /identity/v3/role_assignments?user.id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0&scope.system=all&role.id=896da438cb834959bd42187c944ea2f7 => generated 226 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:16.696538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7009-30ac-4e91-98e2-1026bbbe6999 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:16.697034 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7009-30ac-4e91-98e2-1026bbbe6999 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:16.698044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7009-30ac-4e91-98e2-1026bbbe6999 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:16.702370 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 140/284] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:16.710669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d08627-f9cc-4dd5-92bf-134189f9b707 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:16.711166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d08627-f9cc-4dd5-92bf-134189f9b707 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:16.711647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d08627-f9cc-4dd5-92bf-134189f9b707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:16.747393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-13d08627-f9cc-4dd5-92bf-134189f9b707 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:16.750069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-13d08627-f9cc-4dd5-92bf-134189f9b707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:16.775721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3194e-b005-4758-9aad-7773c36033c2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:16.875591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:16.877211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:16.878042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:16.878843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] PATH_INFO: `/v3/users/de701bb6edb34e0787c75b2445a69e24` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:16.881558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] RBAC: Authorizing `identity:get_user(user_id=de701bb6edb34e0787c75b2445a69e24)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:16.884731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:16.889676 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 145/285] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:16 2022] GET /identity/v3/users/de701bb6edb34e0787c75b2445a69e24 => generated 297 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:16.899866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:16.921744 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 141/286] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:17.179406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:17.197978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dce46-0904-4576-a3ae-5a0747566037 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:17.291604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:17.292797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:17.293262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:17.293745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:17.301929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:17.312681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:17.316370 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 146/287] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:16 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:17.326620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d08627-f9cc-4dd5-92bf-134189f9b707 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:17.450531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:17.550986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:17.617529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d08627-f9cc-4dd5-92bf-134189f9b707 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:17.769599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:17.770717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:17.771203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:17.771672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/de701bb6edb34e0787c75b2445a69e24/roles/fef78cf3dcc64b77a03f0b53167b7430` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:17.782085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a83742d0de534530bf5beab6ecf7fe70, user_id=de701bb6edb34e0787c75b2445a69e24, role_id=fef78cf3dcc64b77a03f0b53167b7430)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:17.785443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:17.833814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650499d-342d-4570-9a5e-fb2ee8d71e48 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:17.845331 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 142/288] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:44:17 2022] PUT /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70/users/de701bb6edb34e0787c75b2445a69e24/roles/fef78cf3dcc64b77a03f0b53167b7430 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:44:17.910714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:17.912314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:17.913106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:17.913944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:17.937843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:17.949612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:17.951694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:17.953552 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:17.957756 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 147/289] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:17.970267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:18.248568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d237069-7aa9-4b53-9ed9-52f7a9e871a1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:18.269310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:18.330364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:18.331880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:18.332708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:18.333403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:18.334985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:18.335820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:18.347581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:18.376539 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 143/290] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:18.390106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:18.469061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:18.655353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75840c7b-9db8-4458-abb1-6289f46e87bd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:18.737956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:18.739116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:18.739616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:18.740106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] PATH_INFO: `/v3/users/139c6db27de04e098a1b7637397cacd7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:18.746371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] RBAC: Authorizing `identity:get_user(user_id=139c6db27de04e098a1b7637397cacd7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:18.749310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:18.752574 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 148/291] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:18 2022] GET /identity/v3/users/139c6db27de04e098a1b7637397cacd7 => generated 297 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:18.762744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:19.050636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456933a-eaa8-4aa8-a2fe-3d92a1ddfd7b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:19.139251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:19.144774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:19.145355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:19.145787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:19.225012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:19.302204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:19.304545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:19.548061 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 144/292] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 786 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:19.557251 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:19.741478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:19.762924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27fa7-af0a-4364-9ec3-5658426bb36c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:19.857936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58171ac5-c951-4fdd-aafe-3d920b88cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:19.859212 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58171ac5-c951-4fdd-aafe-3d920b88cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:19.860030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58171ac5-c951-4fdd-aafe-3d920b88cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:19.860030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58171ac5-c951-4fdd-aafe-3d920b88cfa1 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:19.861624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58171ac5-c951-4fdd-aafe-3d920b88cfa1 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:19.864508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58171ac5-c951-4fdd-aafe-3d920b88cfa1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:19.868265 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 149/293] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:19 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:19.880041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:20.072487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:20.149288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0978d-a160-445e-ab54-79aac178b3e0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:20.233832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35dba7-0923-435b-8976-47ff17315830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:20.235252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35dba7-0923-435b-8976-47ff17315830 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:20.235980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35dba7-0923-435b-8976-47ff17315830 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:20.236709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35dba7-0923-435b-8976-47ff17315830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:20.238355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35dba7-0923-435b-8976-47ff17315830 None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=139c6db27de04e098a1b7637397cacd7) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:20.239099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35dba7-0923-435b-8976-47ff17315830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:20.242362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35dba7-0923-435b-8976-47ff17315830 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:20.255977 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 145/294] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:44:19 2022] GET /identity/v3/role_assignments?user.id=139c6db27de04e098a1b7637397cacd7&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=896da438cb834959bd42187c944ea2f7 => generated 608 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:21.059648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d435b8-7d62-45f6-94f5-ebfc4d951f72 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:21.059648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d435b8-7d62-45f6-94f5-ebfc4d951f72 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:21.059648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d435b8-7d62-45f6-94f5-ebfc4d951f72 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:21.061082 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 150/295] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:21.069323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770923-f16b-4f66-a578-084bf79b5e4f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:21.070830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770923-f16b-4f66-a578-084bf79b5e4f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:21.071591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0770923-f16b-4f66-a578-084bf79b5e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:21.114589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a0770923-f16b-4f66-a578-084bf79b5e4f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:21.116939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a0770923-f16b-4f66-a578-084bf79b5e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:21.178487 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d91dc9-68f0-4b09-81b0-430567e74e4e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:21.179501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d91dc9-68f0-4b09-81b0-430567e74e4e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:21.180271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d91dc9-68f0-4b09-81b0-430567e74e4e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:21.184684 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 151/296] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:21.195603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3b7bf-1fa1-4a5e-b3bc-8201dd923849 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:21.196534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3b7bf-1fa1-4a5e-b3bc-8201dd923849 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:21.197266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3b7bf-1fa1-4a5e-b3bc-8201dd923849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:21.235078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-11d3b7bf-1fa1-4a5e-b3bc-8201dd923849 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:21.237965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11d3b7bf-1fa1-4a5e-b3bc-8201dd923849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:21.276791 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 146/297] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:21.383066 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 152/298] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:21.621448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f466-09a1-4f24-a90f-1b2c683565f0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:21.622491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f466-09a1-4f24-a90f-1b2c683565f0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:21.623272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f466-09a1-4f24-a90f-1b2c683565f0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:21.626591 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 147/299] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:21.636439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a4e98-e397-4c6a-8228-f5c5e808fd71 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:21.637377 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a4e98-e397-4c6a-8228-f5c5e808fd71 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:21.638155 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a4e98-e397-4c6a-8228-f5c5e808fd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:21.671331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-457a4e98-e397-4c6a-8228-f5c5e808fd71 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:21.674002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-457a4e98-e397-4c6a-8228-f5c5e808fd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:21.800754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f466-09a1-4f24-a90f-1b2c683565f0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:21.815706 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 153/300] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:21.903736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a4e98-e397-4c6a-8228-f5c5e808fd71 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:21.935252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:21.948104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:22.032886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f466-09a1-4f24-a90f-1b2c683565f0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:22.113780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:22.117160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.118397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.119105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.143777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a4e98-e397-4c6a-8228-f5c5e808fd71 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:22.146742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:22.159213 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:22.161218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:22.162944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:22.166906 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 148/301] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:22.176926 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:22.222173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:22.223841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.224683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.225382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.227850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:22.230818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:22.233949 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 154/302] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:22.242585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:22.379576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b378b4-1036-44a0-a86e-634bfd9e5aee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:22.410699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:22.418447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:22.438169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cfc68-fc5e-446c-895c-d01542375332 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:22.438431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:22.439414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.439910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.440417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.441706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:22.442377 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:22.445071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:22.467704 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 149/303] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:22.467704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:22.492804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:22.493676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.494186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.495330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] PATH_INFO: `/v3/groups` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.496203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:22.498410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:22.524017 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 155/304] 10.222.0.26 () {60 vars in 1134 bytes} [Sun Sep 25 00:44:22 2022] POST /identity/v3/groups => generated 229 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:44:22.530159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:22.686354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ec8b0-fbc6-4951-8060-9d3e1c01ce51 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:22.706778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286f83-7468-4043-99bb-49026b434fb7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:22.744955 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23362a56-9367-46e0-8d2e-484be53b3af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:22.745828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23362a56-9367-46e0-8d2e-484be53b3af4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.746236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23362a56-9367-46e0-8d2e-484be53b3af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.746647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23362a56-9367-46e0-8d2e-484be53b3af4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.766218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23362a56-9367-46e0-8d2e-484be53b3af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:22.774864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdc29f-eb71-484a-b231-80d24df4cf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-23362a56-9367-46e0-8d2e-484be53b3af4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:22.775813 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:22.779574 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:22.782616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdc29f-eb71-484a-b231-80d24df4cf40 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.782616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdc29f-eb71-484a-b231-80d24df4cf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.782616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdc29f-eb71-484a-b231-80d24df4cf40 None admin] PATH_INFO: `/v3/users/ba7d155b92034d95a74d77f8708eb203` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.783288 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 150/305] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:22.783288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2081e-d4e1-492a-99d1-c6b4d1dc3df2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.783288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2081e-d4e1-492a-99d1-c6b4d1dc3df2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.783288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2081e-d4e1-492a-99d1-c6b4d1dc3df2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.783288 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 151/306] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:22.787842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2081e-d4e1-492a-99d1-c6b4d1dc3df2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:22.815805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdc29f-eb71-484a-b231-80d24df4cf40 None admin] RBAC: Authorizing `identity:get_user(user_id=ba7d155b92034d95a74d77f8708eb203)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:22.818353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdc29f-eb71-484a-b231-80d24df4cf40 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:22.822965 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 156/307] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:22 2022] GET /identity/v3/users/ba7d155b92034d95a74d77f8708eb203 => generated 293 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:22.825311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591763-5ec3-4845-9d73-0cb8a4aa66d5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:22.825744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591763-5ec3-4845-9d73-0cb8a4aa66d5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:22.826194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591763-5ec3-4845-9d73-0cb8a4aa66d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:22.837880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:22.862295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ac591763-5ec3-4845-9d73-0cb8a4aa66d5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:22.864337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ac591763-5ec3-4845-9d73-0cb8a4aa66d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:22.890526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:22.983404 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 157/308] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:22.984519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591763-5ec3-4845-9d73-0cb8a4aa66d5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:22.995637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2081e-d4e1-492a-99d1-c6b4d1dc3df2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:23.064131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:23.065380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:23.065922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:23.066456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:23.067737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:23.068957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:23.071944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:23.089527 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 152/309] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:23.099165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:23.194114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591763-5ec3-4845-9d73-0cb8a4aa66d5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:23.277774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:23.279008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:23.279584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:23.280229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:23.281923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] RBAC: Adding query filter params (role.id=fef78cf3dcc64b77a03f0b53167b7430, user.id=ba7d155b92034d95a74d77f8708eb203, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:23.282567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:23.285392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:23.288908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:23.298518 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 158/310] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Sep 25 00:44:22 2022] GET /identity/v3/role_assignments?user.id=ba7d155b92034d95a74d77f8708eb203&scope.system=all&role.id=fef78cf3dcc64b77a03f0b53167b7430 => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:23.327783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b9572-c57e-4ca4-aeb5-575de74efc5e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:23.347420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:23.397155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:23.398235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:23.398863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:23.399327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] PATH_INFO: `/v3/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:23.401604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] RBAC: Authorizing `identity:get_user(user_id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:23.404195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:23.407530 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 153/311] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:23 2022] GET /identity/v3/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0 => generated 293 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:23.417085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:23.526550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:23.665443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d644-b706-4485-87f6-01d78dfd659a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:23.737734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:23.739416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:23.740274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:23.742146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] PATH_INFO: `/v3/system/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:23.745825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:23.748909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:23.751766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f15d73-bc60-4f32-83d6-904a8861d57f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:23.768128 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 159/312] 10.222.0.26 () {58 vars in 1334 bytes} [Sun Sep 25 00:44:23 2022] PUT /identity/v3/system/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:44:23.828610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:23.830531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:23.831422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:23.832287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:23.856545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:23.867123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:23.868847 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:23.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:23.874938 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 154/313] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:44:23.884418 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:24.028972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:24.116872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c05a2-94c6-4fa6-a991-341d028e2591 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:24.190725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:24.191777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:24.192333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:24.192859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:24.194110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:24.194655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:24.198215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:24.216157 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 160/314] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:24.225378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:24.303643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:24.446658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da2f49-0f56-4e31-855a-bd64329fda99 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:24.517692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:24.518771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:24.519239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:24.519672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] PATH_INFO: `/v3/users/de701bb6edb34e0787c75b2445a69e24` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:24.521780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] RBAC: Authorizing `identity:get_user(user_id=de701bb6edb34e0787c75b2445a69e24)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:24.524666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:24.528725 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 155/315] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:24 2022] GET /identity/v3/users/de701bb6edb34e0787c75b2445a69e24 => generated 297 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:24.537514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:24.757027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0a90b-f891-4f67-9083-69b687738c13 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:24.806741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:24.828017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:24.829072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:24.829636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:24.830785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:24.981776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:24.984659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:25.194868 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 161/316] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:25.203784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:25.407862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6f2e-5365-4472-b366-35d135f7077d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:25.412854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:25.481625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38972c10-b722-4ed6-a0f6-2c4675c96201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:25.482699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38972c10-b722-4ed6-a0f6-2c4675c96201 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:25.483082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38972c10-b722-4ed6-a0f6-2c4675c96201 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:25.483562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38972c10-b722-4ed6-a0f6-2c4675c96201 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:25.485670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38972c10-b722-4ed6-a0f6-2c4675c96201 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:25.488413 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38972c10-b722-4ed6-a0f6-2c4675c96201 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:25.492056 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 156/317] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:25 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:25.501432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:25.552648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:25.721033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11133e89-c3f0-45d7-bc4a-3e9c2767dc21 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:25.796789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40e6c5-b58a-4d1e-a9b9-eba454b74e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:25.797836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40e6c5-b58a-4d1e-a9b9-eba454b74e72 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:25.798356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40e6c5-b58a-4d1e-a9b9-eba454b74e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:25.798856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40e6c5-b58a-4d1e-a9b9-eba454b74e72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:25.800258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40e6c5-b58a-4d1e-a9b9-eba454b74e72 None admin] RBAC: Adding query filter params (role.id=fef78cf3dcc64b77a03f0b53167b7430, scope.project.id=a83742d0de534530bf5beab6ecf7fe70, user.id=de701bb6edb34e0787c75b2445a69e24) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:25.800865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40e6c5-b58a-4d1e-a9b9-eba454b74e72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:25.804040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40e6c5-b58a-4d1e-a9b9-eba454b74e72 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:25.817121 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 162/318] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:44:25 2022] GET /identity/v3/role_assignments?user.id=de701bb6edb34e0787c75b2445a69e24&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=fef78cf3dcc64b77a03f0b53167b7430 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:26.496252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6700-ad57-40a9-8074-0cf432e27d19 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:26.497014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6700-ad57-40a9-8074-0cf432e27d19 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:26.497375 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6700-ad57-40a9-8074-0cf432e27d19 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:26.499792 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 157/319] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:26.508609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baebde6-32ae-4ecb-afeb-f3e6722872b4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:26.509369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baebde6-32ae-4ecb-afeb-f3e6722872b4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:26.509509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baebde6-32ae-4ecb-afeb-f3e6722872b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:26.541991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0baebde6-32ae-4ecb-afeb-f3e6722872b4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:26.544139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0baebde6-32ae-4ecb-afeb-f3e6722872b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:26.683347 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 163/320] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:27.172254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f6700-ad57-40a9-8074-0cf432e27d19 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:27.271678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5675d-0adb-4205-99ad-2c0c87ca6006 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:27.272012 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5675d-0adb-4205-99ad-2c0c87ca6006 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:27.272585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5675d-0adb-4205-99ad-2c0c87ca6006 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:27.274485 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 164/321] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:44:27.281633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1784-b69f-47b0-bedc-e0598ea9b989 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:27.282231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1784-b69f-47b0-bedc-e0598ea9b989 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:27.282711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc1784-b69f-47b0-bedc-e0598ea9b989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:27.318515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-86fc1784-b69f-47b0-bedc-e0598ea9b989 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:27.320857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-86fc1784-b69f-47b0-bedc-e0598ea9b989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:27.354795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:27.402997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f6700-ad57-40a9-8074-0cf432e27d19 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:27.476517 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 165/322] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:27.476757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77724f68-428e-4fc9-8837-b9b09b39bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:27.476757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77724f68-428e-4fc9-8837-b9b09b39bd31 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:27.477277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77724f68-428e-4fc9-8837-b9b09b39bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:27.477644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77724f68-428e-4fc9-8837-b9b09b39bd31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:27.479832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77724f68-428e-4fc9-8837-b9b09b39bd31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:27.482779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77724f68-428e-4fc9-8837-b9b09b39bd31 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:27.488441 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 158/323] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:44:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:27.495532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1784-b69f-47b0-bedc-e0598ea9b989 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:27.739565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc1784-b69f-47b0-bedc-e0598ea9b989 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:27.817657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:27.819289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:27.820145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:27.821031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] PATH_INFO: `/v3/groups` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:27.823442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:27.827108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:27.844748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8a7b0-4268-4df3-8a50-cf8b36f0cb45 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:27.845595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8a7b0-4268-4df3-8a50-cf8b36f0cb45 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:27.845595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8a7b0-4268-4df3-8a50-cf8b36f0cb45 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:27.848202 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 159/324] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:44:27.856562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4b2e-8fa3-4e2c-a187-bd02a33ab0ff None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:27.857007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4b2e-8fa3-4e2c-a187-bd02a33ab0ff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:27.857218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc4b2e-8fa3-4e2c-a187-bd02a33ab0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:27.862345 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 166/325] 10.222.0.26 () {60 vars in 1134 bytes} [Sun Sep 25 00:44:27 2022] POST /identity/v3/groups => generated 226 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:44:27.896018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f8cc4b2e-8fa3-4e2c-a187-bd02a33ab0ff None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:27.898236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f8cc4b2e-8fa3-4e2c-a187-bd02a33ab0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:27.974480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:27.986307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:28.010502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:28.052781 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 160/326] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:28.215611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895aa943-947a-431c-aa5e-7bdfff070f66 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:28.288424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:28.289515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:28.290009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:28.290504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:28.313358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:28.325398 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:28.327128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:28.329678 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 167/327] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:44:28.337297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4b2e-8fa3-4e2c-a187-bd02a33ab0ff None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:28.558415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc4b2e-8fa3-4e2c-a187-bd02a33ab0ff None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:28.574958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:28.644405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:28.644405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:28.644405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:28.644405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:28.644979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:28.644979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:28.646537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:28.667488 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 161/328] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:28.676195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:28.750088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:28.763664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:28.846621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11350cec-00e1-4bb3-b393-f24f73fff0e1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:28.921645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594dbc0-407b-4a44-b9db-25a8112abca0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:28.934231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:28.935342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:28.935841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:28.936339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:28.978334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:28.989342 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:28.991099 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:28.993487 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 168/329] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:44:29.004554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:29.005645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:29.006076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:29.006573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] PATH_INFO: `/v3/users/ba7d155b92034d95a74d77f8708eb203` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:29.006941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:29.009019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] RBAC: Authorizing `identity:get_user(user_id=ba7d155b92034d95a74d77f8708eb203)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:29.012358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:29.016312 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 162/330] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:28 2022] GET /identity/v3/users/ba7d155b92034d95a74d77f8708eb203 => generated 293 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:29.025181 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:29.249414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:29.255356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33c507-df04-431b-87f4-c22ce12b17c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:29.331982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:29.333210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:29.333719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:29.334225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] PATH_INFO: `/v3/system/users/ba7d155b92034d95a74d77f8708eb203/roles/fef78cf3dcc64b77a03f0b53167b7430` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:29.337081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ba7d155b92034d95a74d77f8708eb203, role_id=fef78cf3dcc64b77a03f0b53167b7430)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:29.341991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:29.366400 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 163/331] 10.222.0.26 () {58 vars in 1334 bytes} [Sun Sep 25 00:44:29 2022] PUT /identity/v3/system/users/ba7d155b92034d95a74d77f8708eb203/roles/fef78cf3dcc64b77a03f0b53167b7430 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:44:29.372534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611e09f-4470-473a-bb22-64c1f5caf85e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:29.456745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:29.482424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:29.483486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:29.484002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:29.484542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:29.485892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:29.486455 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:29.489265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:29.513460 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 169/332] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 507 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:29.523008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:29.821861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254abeb-5310-4bc9-a118-559b098313cb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:29.898786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da2a16-4308-4b76-8407-6c08adf2b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:29.900009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da2a16-4308-4b76-8407-6c08adf2b402 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:29.901069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da2a16-4308-4b76-8407-6c08adf2b402 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:29.901544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da2a16-4308-4b76-8407-6c08adf2b402 None admin] PATH_INFO: `/v3/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:29.904065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da2a16-4308-4b76-8407-6c08adf2b402 None admin] RBAC: Authorizing `identity:get_user(user_id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:29.907083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da2a16-4308-4b76-8407-6c08adf2b402 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:29.910946 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 164/333] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:29 2022] GET /identity/v3/users/2ca42fdf49e94bc1b8c9e0e8bc86dcc0 => generated 293 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:29.921064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:30.190399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47802e87-a01e-4a4e-af93-7e9f0810db99 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:30.273775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10642d4b-206f-4c12-8cf9-1aa976d4b2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:30.275434 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642d4b-206f-4c12-8cf9-1aa976d4b2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:30.276231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642d4b-206f-4c12-8cf9-1aa976d4b2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:30.277043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642d4b-206f-4c12-8cf9-1aa976d4b2d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:30.278831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10642d4b-206f-4c12-8cf9-1aa976d4b2d5 None admin] RBAC: Adding query filter params (role.id=896da438cb834959bd42187c944ea2f7, user.id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:30.279736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10642d4b-206f-4c12-8cf9-1aa976d4b2d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:30.282918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10642d4b-206f-4c12-8cf9-1aa976d4b2d5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:30.321292 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 170/334] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Sep 25 00:44:29 2022] GET /identity/v3/role_assignments?user.id=2ca42fdf49e94bc1b8c9e0e8bc86dcc0&scope.system=all&role.id=896da438cb834959bd42187c944ea2f7 => generated 510 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:33.956146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef958-320e-446b-b0d4-318edf51d830 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:33.957842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef958-320e-446b-b0d4-318edf51d830 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:33.958616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ef958-320e-446b-b0d4-318edf51d830 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:33.961954 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 165/335] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:33.972651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a24ba-3647-453c-a0c0-35583e98a10a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:33.974201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a24ba-3647-453c-a0c0-35583e98a10a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:33.975006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a24ba-3647-453c-a0c0-35583e98a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:34.017926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3e5a24ba-3647-453c-a0c0-35583e98a10a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:34.020727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3e5a24ba-3647-453c-a0c0-35583e98a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:34.127640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:34.178934 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 171/336] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:34.260702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648b497-eb0a-4674-960c-8fbb7e2c60d9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:34.261898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648b497-eb0a-4674-960c-8fbb7e2c60d9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:34.262687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648b497-eb0a-4674-960c-8fbb7e2c60d9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:34.265689 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 166/337] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:34.278614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494600b-97cb-4206-9fdc-e9704b81931f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:34.279491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494600b-97cb-4206-9fdc-e9704b81931f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:34.280182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494600b-97cb-4206-9fdc-e9704b81931f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:34.327855 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a494600b-97cb-4206-9fdc-e9704b81931f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:34.329828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a494600b-97cb-4206-9fdc-e9704b81931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:34.515907 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 172/338] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:34.709050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249994-7b5c-48b4-a9fd-62a6e72b326d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:34.709563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249994-7b5c-48b4-a9fd-62a6e72b326d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:34.710023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249994-7b5c-48b4-a9fd-62a6e72b326d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:34.713251 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 167/339] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:34.722691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55e36-de89-4793-9110-1ee40022448c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:34.723214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55e36-de89-4793-9110-1ee40022448c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:34.723713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c55e36-de89-4793-9110-1ee40022448c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:34.772207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-20c55e36-de89-4793-9110-1ee40022448c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:34.774691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-20c55e36-de89-4793-9110-1ee40022448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:34.945935 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 173/340] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:35.085112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74249994-7b5c-48b4-a9fd-62a6e72b326d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:35.163817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:35.341411 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74249994-7b5c-48b4-a9fd-62a6e72b326d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:35.411059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:35.412163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:35.412687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:35.413176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:35.434006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:35.444774 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:35.446549 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:35.449712 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 168/341] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:35.458364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55e36-de89-4793-9110-1ee40022448c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:35.554066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:35.630782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:35.732915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c55e36-de89-4793-9110-1ee40022448c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:35.853698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:35.854703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:35.855170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:35.855640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:35.856938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:35.857471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:35.860136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:35.881089 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 174/342] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:35.882938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:35.922958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:35.933944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a027b2e-90f7-4f95-88e9-589a959ebb6c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:36.034807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:36.035853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.036273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.036740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.066816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:36.078817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:36.080864 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:36.082630 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:36.087324 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 169/343] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:44:36.094964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:36.158913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:36.165571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42bae1-412c-4e2c-ac53-8ce525b13399 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:36.254290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:36.255649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.261835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.262407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.289464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:36.301698 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:36.303367 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 175/344] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:36.305008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:36.347785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d327-a2d7-4231-8eea-be777db45d56 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:36.423383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1297c5-a11e-447e-81da-b26eb5dd037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:36.425000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1297c5-a11e-447e-81da-b26eb5dd037a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.425482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1297c5-a11e-447e-81da-b26eb5dd037a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.426211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1297c5-a11e-447e-81da-b26eb5dd037a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.469423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:36.531514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feefd6-da83-42c4-a444-aa56fb2e56c3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:36.573384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1297c5-a11e-447e-81da-b26eb5dd037a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:36.575384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1297c5-a11e-447e-81da-b26eb5dd037a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:36.603307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a557a-7683-40f4-bfb8-d2648e01b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:36.604743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a557a-7683-40f4-bfb8-d2648e01b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.605083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a557a-7683-40f4-bfb8-d2648e01b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.606040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a557a-7683-40f4-bfb8-d2648e01b8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.607045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a557a-7683-40f4-bfb8-d2648e01b8a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:36.607605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a557a-7683-40f4-bfb8-d2648e01b8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:36.610396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a557a-7683-40f4-bfb8-d2648e01b8a2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:36.629656 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 176/345] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:36.632058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb4c9b-d94d-4e8d-b450-6952ed4a6f3f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.632620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb4c9b-d94d-4e8d-b450-6952ed4a6f3f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.633119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb4c9b-d94d-4e8d-b450-6952ed4a6f3f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.636024 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 177/346] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:36.638682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6f70c-1a24-4a9a-bbb6-993d5a497884 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.639628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6f70c-1a24-4a9a-bbb6-993d5a497884 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.640452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6f70c-1a24-4a9a-bbb6-993d5a497884 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.643386 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 178/347] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:36.645380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6f70c-1a24-4a9a-bbb6-993d5a497884 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:36.693878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:36.783076 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 170/348] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:36.787503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571d583-414a-4f07-9ada-363e3756feed None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.787503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571d583-414a-4f07-9ada-363e3756feed None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.787503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571d583-414a-4f07-9ada-363e3756feed None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.789591 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 171/349] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:36.790831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571d583-414a-4f07-9ada-363e3756feed None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:36.844466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6f70c-1a24-4a9a-bbb6-993d5a497884 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:36.897841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:36.905623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c28fc5-7f65-4716-9937-2d3fd8f3af43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:36.906722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28fc5-7f65-4716-9937-2d3fd8f3af43 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.907193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28fc5-7f65-4716-9937-2d3fd8f3af43 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.907621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28fc5-7f65-4716-9937-2d3fd8f3af43 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.908714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c28fc5-7f65-4716-9937-2d3fd8f3af43 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:36.909213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c28fc5-7f65-4716-9937-2d3fd8f3af43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:36.911553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c28fc5-7f65-4716-9937-2d3fd8f3af43 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:36.928163 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 179/350] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:36.930148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405bdd6-ede6-4d89-a891-d6f856a4d807 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:36.930527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405bdd6-ede6-4d89-a891-d6f856a4d807 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:36.930928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405bdd6-ede6-4d89-a891-d6f856a4d807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:36.960033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d405bdd6-ede6-4d89-a891-d6f856a4d807 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:36.962084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d405bdd6-ede6-4d89-a891-d6f856a4d807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:37.003692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571d583-414a-4f07-9ada-363e3756feed None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:37.071285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b94c7-6967-4e69-8dad-a0180ba2abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:37.072870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b94c7-6967-4e69-8dad-a0180ba2abfd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:37.073228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b94c7-6967-4e69-8dad-a0180ba2abfd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:37.073649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b94c7-6967-4e69-8dad-a0180ba2abfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:37.091589 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 180/351] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:37.093828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eff92b-e02a-43f1-a057-e85c442c3c04 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:37.094348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eff92b-e02a-43f1-a057-e85c442c3c04 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:37.094860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eff92b-e02a-43f1-a057-e85c442c3c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:37.130369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f9eff92b-e02a-43f1-a057-e85c442c3c04 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:37.133623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f9eff92b-e02a-43f1-a057-e85c442c3c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:37.159295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:37.231696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b94c7-6967-4e69-8dad-a0180ba2abfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:37.236362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b94c7-6967-4e69-8dad-a0180ba2abfd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:37.289077 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 181/352] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:37.291303 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eff92b-e02a-43f1-a057-e85c442c3c04 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:37.367898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:37.502629 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 172/353] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:37.505012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c95eaf-d9d2-45fc-8a93-91750e9cb3bd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:37.505498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c95eaf-d9d2-45fc-8a93-91750e9cb3bd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:37.505984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c95eaf-d9d2-45fc-8a93-91750e9cb3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:37.536045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eff92b-e02a-43f1-a057-e85c442c3c04 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:37.545565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-48c95eaf-d9d2-45fc-8a93-91750e9cb3bd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:37.547439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-48c95eaf-d9d2-45fc-8a93-91750e9cb3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:37.600522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:37.602176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:37.602979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:37.604279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:37.606837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:37.609745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:37.613090 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 182/354] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:37 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:37.614345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:37.651214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:37.696990 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 173/355] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:37.699134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c95eaf-d9d2-45fc-8a93-91750e9cb3bd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:37.868434 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35c062-298c-44ba-8b95-a48bea5ae784 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:37.880302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:37.921275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c95eaf-d9d2-45fc-8a93-91750e9cb3bd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:37.949423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:37.950264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:37.950432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:37.950613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:38.003039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:38.004152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:38.004709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:38.005178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:38.008414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:38.012049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:38.020157 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 174/356] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:37 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:38.020157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:38.112969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:38.115784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:38.145083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:38.234636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcaf4d-b104-4020-a1ae-bc22b673dfb2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:38.302502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:38.303497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:38.303876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:38.304404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:38.325184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:38.327497 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 183/357] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:38.329035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:38.334768 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:38.336756 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:38.338439 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 175/358] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:38.339747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:38.359149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:38.526013 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de98c1-92e6-4beb-a550-72bffe733662 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:38.534300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b1e27-0f11-4b44-b386-4edc2320e7d8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:38.552537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:38.603334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:38.604787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:38.605109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:38.605438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:38.609781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:38.610853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:38.611350 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:38.611844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:38.617215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:38.620596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:38.623689 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 184/359] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:38 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:38.623689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:38.637389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:38.645565 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:38.646950 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 25 00:44:38.648350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:38.649527 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 176/360] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:38.651081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:38.767940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:38.790584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561280e5-d2c2-4ebd-a0be-eddf3ba3e01b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:38.819784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f5c02-f874-4fcf-9a88-2e8021e79d38 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:38.835902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:38.836732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:38.837101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:38.837525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:38.839554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:38.841348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:38.843243 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 185/361] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:38 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:38.844645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:38.867265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:38.868346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:38.868852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:38.869875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:38.889187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:38.898383 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:38.904757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 177/362] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:44:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:38.907062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:38.909021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:39.002590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448b115-8fec-4baa-a8bb-70de021a41d9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.050061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf972cad-47f5-44a3-9a77-7acdc254aca1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.070475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.071468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.071791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.072810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:39.074987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:39.077861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:39.081051 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 186/363] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:38 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:39.082478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:39.102168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:39.114857 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.115902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.116311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.116775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:39.117973 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:39.118433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:39.120104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:39.133752 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 178/364] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:39.136140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:39.267541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f569c26-7664-4704-9162-55dd189f2f97 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.293022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:39.327967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53bbec-a40e-4139-9458-70ceba3764b9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.338097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.339054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.339477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.339922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:39.341214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=4010acd13d9445b1943eac27cb61aaa2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:39.341681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:39.343533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:39.350879 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 187/365] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:39 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=896da438cb834959bd42187c944ea2f7 => generated 260 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:39.351771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:39.397553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.398659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.399165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.399673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:39.401121 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:39.401664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:39.404497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:39.421870 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 179/366] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:39.428965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:39.519471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:39.530539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160950c-a09e-430a-beec-2af555c211bf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.602183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.603344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.603995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.604594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:39.606090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=4010acd13d9445b1943eac27cb61aaa2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:39.606730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:39.609538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365b995-b1c3-456f-97f4-1b905d4cc7ee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.610026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:39.619909 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 188/367] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:39 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 260 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:39.621246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:39.677781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.678428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.678814 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.679146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:39.680072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:39.680637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:39.683379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:39.706863 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 180/368] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:39.706863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:39.734571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:39.857462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610d611-1e95-4740-8f95-a1a720108763 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.889908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8db74-a36d-47f2-a9d3-c604883059f8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:39.925353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.926466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.926967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.927461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:39.928228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:39.929644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:39.932325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:39.935529 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 189/369] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:39 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:39.937147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:39.972836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:39.973482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:39.974195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:39.974484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:40.135734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:40.138861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:40.174330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07b2bd-22bc-4e78-8ce4-da4db3281b30 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:40.193620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:40.249401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:40.250307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:40.250951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:40.251620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] PATH_INFO: `/v3/users/ba7d155b92034d95a74d77f8708eb203` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:40.254210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] RBAC: Authorizing `identity:get_user(user_id=ba7d155b92034d95a74d77f8708eb203)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:40.257348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:40.260759 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 190/370] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:44:39 2022] GET /identity/v3/users/ba7d155b92034d95a74d77f8708eb203 => generated 293 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:40.262774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:40.354449 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 181/371] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:40.356867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:40.393850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:40.461242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45da43f-a770-4061-a02d-a46149add5f2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:40.530886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:40.531896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:40.532183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:40.532559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:40.566273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74f18f-3de6-4635-91b9-eaacb8463bd7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:40.617847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:40.640135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:40.641794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:40.642621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:40.643357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:40.645219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=a83742d0de534530bf5beab6ecf7fe70) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:40.646091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:40.649295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:40.660958 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 182/372] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:40 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=896da438cb834959bd42187c944ea2f7 => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:40.662920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:40.666723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:40.669480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:40.840592 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 191/373] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:40.841531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:40.898572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:40.940708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c4e39-e991-4ffc-8736-00baf9192d4b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:41.010854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:41.012626 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:41.013392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:41.014151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:41.015927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] RBAC: Adding query filter params (role.id=fef78cf3dcc64b77a03f0b53167b7430, user.id=ba7d155b92034d95a74d77f8708eb203, scope.system=all) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:41.016800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:41.019622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:41.044309 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 183/374] 10.222.0.26 () {56 vars in 1308 bytes} [Sun Sep 25 00:44:40 2022] GET /identity/v3/role_assignments?user.id=ba7d155b92034d95a74d77f8708eb203&scope.system=all&role.id=fef78cf3dcc64b77a03f0b53167b7430 => generated 510 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:41.046393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:41.073231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:41.171142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaaec9-5c1a-4d1e-82e6-20546565edb5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:41.253716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66935ef0-94c9-460c-8159-4b588fbe7fc4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:41.297955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:41.299018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:41.299485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:41.299966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:41.302241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:41.308204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:41.313908 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 192/375] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:40 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:41.330002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:41.333951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:41.335126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:41.335607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:41.336189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] PATH_INFO: `/v3/projects/3708ba2402cc4ee7abd81255a285d2b9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:41.340036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] RBAC: Authorizing `identity:get_project(project_id=3708ba2402cc4ee7abd81255a285d2b9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:41.346017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:41.349369 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 184/376] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:41 2022] GET /identity/v3/projects/3708ba2402cc4ee7abd81255a285d2b9 => generated 344 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:41.360178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:41.377754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:41.615283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b1cce-9127-4c5b-9ff7-189ae09a3fed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:41.615720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca73905-4b7f-403e-a295-8ae63f712837 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:41.728937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:41.731938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:41.733539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:41.734307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:41.736969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:41.739980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:41.747748 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 193/377] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:41 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:41.764593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:41.777562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:41.779925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:41.780553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:41.781058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] PATH_INFO: `/v3/groups/fa41eed58c8e4624ad28ba89d255c4b5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:41.807711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] RBAC: Authorizing `identity:get_group(group_id=fa41eed58c8e4624ad28ba89d255c4b5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:41.810690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:41.813983 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 185/378] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:41 2022] GET /identity/v3/groups/fa41eed58c8e4624ad28ba89d255c4b5 => generated 229 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:41.824155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:41.828871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:41.985793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:42.057657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d36fe3-1dae-4760-920f-6123d1f60ef9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:42.061539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ff19-ca90-4616-b75b-4d5975f1a7fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:42.162577 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf235bd-112b-49d9-a253-26dcb509d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:42.165684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf235bd-112b-49d9-a253-26dcb509d8af None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:42.167395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf235bd-112b-49d9-a253-26dcb509d8af None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:42.170403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf235bd-112b-49d9-a253-26dcb509d8af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:42.172239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf235bd-112b-49d9-a253-26dcb509d8af None admin] RBAC: Adding query filter params (group.id=fa41eed58c8e4624ad28ba89d255c4b5, role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=3708ba2402cc4ee7abd81255a285d2b9) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:42.173516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf235bd-112b-49d9-a253-26dcb509d8af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:42.179536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf235bd-112b-49d9-a253-26dcb509d8af None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:42.192455 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c85b3-d1d6-448c-8ccb-bef42f80f863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:42.194066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c85b3-d1d6-448c-8ccb-bef42f80f863 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:42.198787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c85b3-d1d6-448c-8ccb-bef42f80f863 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:42.199696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c85b3-d1d6-448c-8ccb-bef42f80f863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:42.201631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c85b3-d1d6-448c-8ccb-bef42f80f863 None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=a83742d0de534530bf5beab6ecf7fe70) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:42.206382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c85b3-d1d6-448c-8ccb-bef42f80f863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:42.210208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c85b3-d1d6-448c-8ccb-bef42f80f863 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:42.210638 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 186/379] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:41 2022] GET /identity/v3/role_assignments?group.id=fa41eed58c8e4624ad28ba89d255c4b5&scope.project.id=3708ba2402cc4ee7abd81255a285d2b9&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 260 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:42.233813 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 194/380] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:41 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 260 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:43.346048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ab840-83aa-422e-a554-3817e84bdb7e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:43.346805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ab840-83aa-422e-a554-3817e84bdb7e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:43.346937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ab840-83aa-422e-a554-3817e84bdb7e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:43.349386 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 187/381] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:43.358119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb55c3-6204-43a4-b331-3901b2402f44 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:43.358927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb55c3-6204-43a4-b331-3901b2402f44 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:43.359247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb55c3-6204-43a4-b331-3901b2402f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:43.401622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cdeb55c3-6204-43a4-b331-3901b2402f44 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:43.403828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cdeb55c3-6204-43a4-b331-3901b2402f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:43.557629 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 195/382] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:43.974018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e26b3-6e4b-4055-935f-df1e14dc375e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:43.974706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e26b3-6e4b-4055-935f-df1e14dc375e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:43.975267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e26b3-6e4b-4055-935f-df1e14dc375e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:43.977555 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 188/383] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:43.986416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272f788-e5fa-4fbe-90e7-feb3f92a11ad None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:43.986840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272f788-e5fa-4fbe-90e7-feb3f92a11ad None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:43.987141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272f788-e5fa-4fbe-90e7-feb3f92a11ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:44.010098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:44.018318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d272f788-e5fa-4fbe-90e7-feb3f92a11ad None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:44.020817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d272f788-e5fa-4fbe-90e7-feb3f92a11ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:44.123545 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 196/384] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:44.522567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e26b3-6e4b-4055-935f-df1e14dc375e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:44.782590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e26b3-6e4b-4055-935f-df1e14dc375e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:44.855617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272f788-e5fa-4fbe-90e7-feb3f92a11ad None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:44.873074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:44.874326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:44.874980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:44.875493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:44.880925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:44.913527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:44.922944 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:44.924591 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:44.926065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:44.938521 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 189/385] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:44.957870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:45.065078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272f788-e5fa-4fbe-90e7-feb3f92a11ad None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:45.137342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:45.138447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:45.138930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:45.139427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:45.160639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:45.167131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:45.188187 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:45.190034 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:45.191572 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 197/386] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:45.210639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:45.407614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198661b-0a21-4271-8b37-90f42c32ed53 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:45.431370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03282b9c-8e00-42a9-b1d5-4ad3266622fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:45.500907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:45.501915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:45.502405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:45.502881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:45.504113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:45.504678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:45.507271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:45.524739 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 198/387] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:45.534936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:45.545675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:45.546730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:45.547224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:45.547573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:45.548279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:45.548852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:45.551522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:45.582677 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 190/388] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 626 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:45.604888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:45.673720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:45.726669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:45.772510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a6164-9fc7-4869-ab68-8e11f44790fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:45.838334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aac682-e686-46ca-91ca-ba956cedb84e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:45.850863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec48f1a-6dae-4801-aedb-38ccdef6d79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:45.852582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48f1a-6dae-4801-aedb-38ccdef6d79a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:45.854050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48f1a-6dae-4801-aedb-38ccdef6d79a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:45.854847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48f1a-6dae-4801-aedb-38ccdef6d79a None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:45.857616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec48f1a-6dae-4801-aedb-38ccdef6d79a None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:45.861870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec48f1a-6dae-4801-aedb-38ccdef6d79a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:45.870959 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 199/389] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:45 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:45.873793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7eb4b-5917-4c49-92b3-7116eae9d4bc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:45.874652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7eb4b-5917-4c49-92b3-7116eae9d4bc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:45.875475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7eb4b-5917-4c49-92b3-7116eae9d4bc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:45.882583 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 200/390] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:45.885480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7eb4b-5917-4c49-92b3-7116eae9d4bc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:45.904829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078bb91-67dd-4c16-b923-4be7a813006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:45.906359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078bb91-67dd-4c16-b923-4be7a813006b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:45.906559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078bb91-67dd-4c16-b923-4be7a813006b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:45.906936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078bb91-67dd-4c16-b923-4be7a813006b None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:45.909021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078bb91-67dd-4c16-b923-4be7a813006b None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:45.911904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078bb91-67dd-4c16-b923-4be7a813006b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:45.915588 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 191/391] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:45 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:45.923974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1bfa4-eaf0-4945-82e3-c11ecc065214 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:45.924550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1bfa4-eaf0-4945-82e3-c11ecc065214 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:45.925048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1bfa4-eaf0-4945-82e3-c11ecc065214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:45.960851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6fd1bfa4-eaf0-4945-82e3-c11ecc065214 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:45.963160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6fd1bfa4-eaf0-4945-82e3-c11ecc065214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:46.102262 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 192/392] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:46.104587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1bfa4-eaf0-4945-82e3-c11ecc065214 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:46.112903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7eb4b-5917-4c49-92b3-7116eae9d4bc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:46.171467 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:46.211047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:46.213360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:46.214208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:46.215002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:46.221895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:46.222865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:46.227334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:46.239258 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 201/393] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:45 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:46.242484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:46.386279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1bfa4-eaf0-4945-82e3-c11ecc065214 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:46.461013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:46.461813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e90e4-3183-41af-b307-dfaf257d3ae7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:46.462191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:46.463210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:46.463794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:46.466017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:46.469989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:46.473103 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 193/394] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:46 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:46.483003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:46.538190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660611f7-e95e-4f07-bb36-cb13d0064309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:46.539868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660611f7-e95e-4f07-bb36-cb13d0064309 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:46.540973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660611f7-e95e-4f07-bb36-cb13d0064309 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:46.541741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660611f7-e95e-4f07-bb36-cb13d0064309 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2/groups/2555bae83c2141ea992f0bab7d6601de/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:46.546036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660611f7-e95e-4f07-bb36-cb13d0064309 None admin] RBAC: Authorizing `identity:create_grant(project_id=4010acd13d9445b1943eac27cb61aaa2, group_id=2555bae83c2141ea992f0bab7d6601de, role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:46.549530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660611f7-e95e-4f07-bb36-cb13d0064309 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:46.596729 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 202/395] 10.222.0.26 () {58 vars in 1442 bytes} [Sun Sep 25 00:44:46 2022] PUT /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2/groups/2555bae83c2141ea992f0bab7d6601de/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:44:46.670570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28678eb-5f5c-429d-9d80-c92c090eac9e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:46.671520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28678eb-5f5c-429d-9d80-c92c090eac9e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:46.672284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28678eb-5f5c-429d-9d80-c92c090eac9e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:46.676045 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 203/396] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:46.686231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdd2e4-b70b-40e2-86f9-da04fb9a409f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:46.687197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdd2e4-b70b-40e2-86f9-da04fb9a409f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:46.687986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdd2e4-b70b-40e2-86f9-da04fb9a409f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:46.699112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510e3e-3373-41b9-a453-d8bc898a7740 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:46.756972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:46.766292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d9bdd2e4-b70b-40e2-86f9-da04fb9a409f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:46.769158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d9bdd2e4-b70b-40e2-86f9-da04fb9a409f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:46.772600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:46.780721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:46.780721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:46.780721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2/groups/2555bae83c2141ea992f0bab7d6601de/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:46.780721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] RBAC: Authorizing `identity:create_grant(project_id=4010acd13d9445b1943eac27cb61aaa2, group_id=2555bae83c2141ea992f0bab7d6601de, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:46.781761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:46.811588 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 194/397] 10.222.0.26 () {58 vars in 1442 bytes} [Sun Sep 25 00:44:46 2022] PUT /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2/groups/2555bae83c2141ea992f0bab7d6601de/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:44:46.912884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:46.948812 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 204/398] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:46.959163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:47.149046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317149c-0166-4d32-be4f-b3a7e34364a2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:47.150096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317149c-0166-4d32-be4f-b3a7e34364a2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:47.151414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317149c-0166-4d32-be4f-b3a7e34364a2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:47.157023 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 205/399] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:47.174297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96a68-b814-47a8-b761-f2b05fdf7d2f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:47.175264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96a68-b814-47a8-b761-f2b05fdf7d2f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:47.176040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96a68-b814-47a8-b761-f2b05fdf7d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:47.178243 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4471cb1-9801-4f53-b910-9d204a295279 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:47.213891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-66d96a68-b814-47a8-b761-f2b05fdf7d2f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:47.216522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-66d96a68-b814-47a8-b761-f2b05fdf7d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:47.249222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:47.250403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:47.250916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:47.251412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:47.278568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:47.293538 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:47.294943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:47.296352 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 195/400] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:47.317814 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:47.383478 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 206/401] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:47.421763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:47.549395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96a68-b814-47a8-b761-f2b05fdf7d2f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:47.551178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816ff2-07d9-4374-9420-5c30e50a241d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:47.623241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:47.624742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:47.625472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:47.626159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:47.627637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:47.628435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:47.631230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:47.650501 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 196/402] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:47.660528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:47.858664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d96a68-b814-47a8-b761-f2b05fdf7d2f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:47.882860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7ef58-baaa-4325-934a-c9c8a8c255c9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:47.890329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:47.918660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:47.959655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:47.961462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:47.962261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:47.963639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:47.966211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:47.969282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:47.973353 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 197/403] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:47 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:47.981929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:48.000715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:48.002370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:48.003172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:48.003944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:48.035371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:48.047672 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:48.049549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:48.051189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:48.056541 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 207/404] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 508 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:44:48.069267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:48.286237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d8090-2637-4eb4-94d7-b0192042a43a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:48.307844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec39d21-d690-45b9-b88a-176497d34cbc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:48.380968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:48.382220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:48.384250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:48.385333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:48.388408 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:48.391433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:48.395761 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 198/405] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:47 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:48.397530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:48.399133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:48.401528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:48.403504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:48.405186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:48.405810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:48.406185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:48.407985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:48.428161 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 208/406] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:48.429844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:48.492763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:48.508275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:48.610997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf781cc-728f-40ac-9ab0-6ba15ff64d74 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:48.622606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae155a-7492-4338-ac2d-f18ba6c274d4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:48.667109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:48.667449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:48.667659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:48.667939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:48.684127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:48.689836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:48.690979 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:48.692093 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:48.692974 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 199/407] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:48.694474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:48.695216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:48.695798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70/groups/2555bae83c2141ea992f0bab7d6601de/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:48.698320 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:48.699221 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] RBAC: Authorizing `identity:create_grant(project_id=a83742d0de534530bf5beab6ecf7fe70, group_id=2555bae83c2141ea992f0bab7d6601de, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:48.702516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:48.728882 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 209/408] 10.222.0.26 () {58 vars in 1442 bytes} [Sun Sep 25 00:44:48 2022] PUT /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70/groups/2555bae83c2141ea992f0bab7d6601de/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:44:48.730061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:48.883081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:48.892848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ff617-206b-4d85-9093-d18be2444835 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:48.930466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:48.934681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cee47-1337-4009-a5fc-962cd01f18dc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:49.015669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:49.015949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:49.017242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:49.017633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:49.017948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:49.018506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:49.018670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:49.019290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] PATH_INFO: `/v3/groups/fa41eed58c8e4624ad28ba89d255c4b5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:49.020145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:49.020800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:49.022049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] RBAC: Authorizing `identity:get_group(group_id=fa41eed58c8e4624ad28ba89d255c4b5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:49.023606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:49.032418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:49.036041 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 200/409] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:48 2022] GET /identity/v3/groups/fa41eed58c8e4624ad28ba89d255c4b5 => generated 229 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:49.040694 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 210/410] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:49.045853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:49.051130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:49.253083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27c92-44f1-47a7-82ff-7e7654bbea31 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:49.303837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ca768-0864-481b-9802-c14fa48cdc69 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:49.334134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:49.335340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:49.335933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:49.337233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:49.339861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:49.346999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:49.356139 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 211/411] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:49 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:49.361244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:49.389975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:49.391484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:49.392239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:49.392911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] PATH_INFO: `/v3/projects/3708ba2402cc4ee7abd81255a285d2b9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:49.400862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] RBAC: Authorizing `identity:get_project(project_id=3708ba2402cc4ee7abd81255a285d2b9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:49.404296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:49.417296 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 201/412] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:49 2022] GET /identity/v3/projects/3708ba2402cc4ee7abd81255a285d2b9 => generated 344 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:49.422531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:49.426391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:49.582612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e9f5a-0d90-4343-9bea-27a17a33daaa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:49.614665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:49.658974 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:49.660752 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:49.661593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:49.662349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:49.665081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:49.669107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:49.680571 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 212/413] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:49 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:49.683814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:49.840174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb3ff8-6f1e-4942-af89-439723883881 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:49.891618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:49.928547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51858252-dd33-450d-a2cf-c7545964ad45 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:49.959362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f94f6a-5940-40d2-888c-6bba531edd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:49.960561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94f6a-5940-40d2-888c-6bba531edd95 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:49.961054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94f6a-5940-40d2-888c-6bba531edd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:49.961533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94f6a-5940-40d2-888c-6bba531edd95 None admin] PATH_INFO: `/v3/projects/3708ba2402cc4ee7abd81255a285d2b9/groups/fa41eed58c8e4624ad28ba89d255c4b5/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:49.965732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f94f6a-5940-40d2-888c-6bba531edd95 None admin] RBAC: Authorizing `identity:create_grant(project_id=3708ba2402cc4ee7abd81255a285d2b9, group_id=fa41eed58c8e4624ad28ba89d255c4b5, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:49.968342 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f94f6a-5940-40d2-888c-6bba531edd95 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:49.996305 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 202/414] 10.222.0.26 () {58 vars in 1442 bytes} [Sun Sep 25 00:44:49 2022] PUT /identity/v3/projects/3708ba2402cc4ee7abd81255a285d2b9/groups/fa41eed58c8e4624ad28ba89d255c4b5/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:44:50.063356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376cab1-0983-4d89-aaa8-f9ca4eba71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:50.065733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376cab1-0983-4d89-aaa8-f9ca4eba71db None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:50.066667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376cab1-0983-4d89-aaa8-f9ca4eba71db None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:50.067523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376cab1-0983-4d89-aaa8-f9ca4eba71db None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70/groups/2555bae83c2141ea992f0bab7d6601de/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:50.075894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376cab1-0983-4d89-aaa8-f9ca4eba71db None admin] RBAC: Authorizing `identity:create_grant(project_id=a83742d0de534530bf5beab6ecf7fe70, group_id=2555bae83c2141ea992f0bab7d6601de, role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:50.080151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376cab1-0983-4d89-aaa8-f9ca4eba71db None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:50.120930 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 213/415] 10.222.0.26 () {58 vars in 1442 bytes} [Sun Sep 25 00:44:49 2022] PUT /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70/groups/2555bae83c2141ea992f0bab7d6601de/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:44:50.972271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cddb2f-7e8a-467f-ba87-e2c114741eab None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:50.973900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cddb2f-7e8a-467f-ba87-e2c114741eab None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:50.974700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cddb2f-7e8a-467f-ba87-e2c114741eab None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:50.978366 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 203/416] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:50.988189 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71a962-4b05-4535-825f-c17af9c2f9dd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:50.989203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71a962-4b05-4535-825f-c17af9c2f9dd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:50.990000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71a962-4b05-4535-825f-c17af9c2f9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:51.039054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0a71a962-4b05-4535-825f-c17af9c2f9dd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:51.041807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0a71a962-4b05-4535-825f-c17af9c2f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:51.199320 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 214/417] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:51.420563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d56d4-8736-4b04-8434-26837bbfb089 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:51.421073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d56d4-8736-4b04-8434-26837bbfb089 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:51.421550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d56d4-8736-4b04-8434-26837bbfb089 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:51.426204 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 204/418] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:51.435807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740070-080b-4491-9c1a-b7ad30f49502 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:51.436319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740070-080b-4491-9c1a-b7ad30f49502 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:51.436884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d740070-080b-4491-9c1a-b7ad30f49502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:51.480795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0d740070-080b-4491-9c1a-b7ad30f49502 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:51.482569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0d740070-080b-4491-9c1a-b7ad30f49502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:51.663313 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 215/419] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:51.783315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d56d4-8736-4b04-8434-26837bbfb089 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:52.025075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:52.036535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d56d4-8736-4b04-8434-26837bbfb089 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:52.123755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:52.124901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:52.125480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:52.125961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:52.154126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:52.164632 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:52.167078 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:52.169439 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 205/420] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:52.179741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d740070-080b-4491-9c1a-b7ad30f49502 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:52.231563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:52.384558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:52.460247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d740070-080b-4491-9c1a-b7ad30f49502 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:52.536166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:52.537743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:52.538552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:52.539309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:52.541330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:52.542218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:52.545092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:52.568955 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 216/421] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:52.574246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:52.743303 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:52.801269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486795a-e9da-4725-926b-a3a49ec5e077 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:52.804062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe240a-297c-41ee-b4a7-17d5b77399ca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:52.876990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:52.878573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:52.879440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:52.881216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:52.961630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:52.963202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:52.963663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:52.964147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:53.018179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:53.034031 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:53.036610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:53.038520 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:53.044614 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 206/422] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 661 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:53.049775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:53.052935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:53.056106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:53.123865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:53.229940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:53.289067 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 217/423] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 25 00:44:53.296416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:53.450514 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c5e93-9b67-4a9e-b889-49b8dc77b23a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:53.580820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:53.582714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:53.583850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:53.584631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:53.586458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:53.587301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:53.591518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:53.616561 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 207/424] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 561 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:53.619563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938675ee-24ca-4ad9-87d0-5561cbd02c6b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:53.625424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:53.696144 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:53.697767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:53.698772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:53.699558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:53.702414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:53.705705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:53.716551 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 218/425] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:53 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:53.721500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:53.828562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:53.889528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:53.943233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b483fb7-0eae-4aa6-8fb8-fa225b4d6474 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:53.952370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfb79-4cd0-441d-8211-d2811808b0e7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:54.011259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36946769-6ecc-43c5-a0b5-08b798c50ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:54.012628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946769-6ecc-43c5-a0b5-08b798c50ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.013102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946769-6ecc-43c5-a0b5-08b798c50ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.013581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36946769-6ecc-43c5-a0b5-08b798c50ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.022032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bb509-edb1-4624-93f0-bfdc10982f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:54.023148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb509-edb1-4624-93f0-bfdc10982f8f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.023615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb509-edb1-4624-93f0-bfdc10982f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.024102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb509-edb1-4624-93f0-bfdc10982f8f None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.026941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bb509-edb1-4624-93f0-bfdc10982f8f None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:54.029855 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bb509-edb1-4624-93f0-bfdc10982f8f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:54.033488 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 219/426] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:53 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:54.035784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef390e63-3ad0-412c-ad62-16b471bfebe1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.036234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef390e63-3ad0-412c-ad62-16b471bfebe1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.036760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef390e63-3ad0-412c-ad62-16b471bfebe1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.039129 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 220/427] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:54.045867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef390e63-3ad0-412c-ad62-16b471bfebe1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:54.154993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36946769-6ecc-43c5-a0b5-08b798c50ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:54.158796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36946769-6ecc-43c5-a0b5-08b798c50ab7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:54.254440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef390e63-3ad0-412c-ad62-16b471bfebe1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:54.284260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:54.313380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:54.349375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:54.351638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.352371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.353115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.355331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=4010acd13d9445b1943eac27cb61aaa2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:54.356151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:54.364418 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 208/428] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 25 00:44:54.364418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d350ee8-5107-4735-947a-580c3ba2a829 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.364418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d350ee8-5107-4735-947a-580c3ba2a829 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.364418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d350ee8-5107-4735-947a-580c3ba2a829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.367177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:54.380285 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 221/429] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:54 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 611 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:54.382197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:54.393637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4d350ee8-5107-4735-947a-580c3ba2a829 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:54.395707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4d350ee8-5107-4735-947a-580c3ba2a829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:54.532958 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 209/430] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:54.604421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10abab1-d677-4a5a-8219-79532bec3e85 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:54.676364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b69a68-7948-4c7a-b4a1-0937a110e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:54.677903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69a68-7948-4c7a-b4a1-0937a110e644 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.678654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69a68-7948-4c7a-b4a1-0937a110e644 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.679338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69a68-7948-4c7a-b4a1-0937a110e644 None admin] PATH_INFO: `/v3/projects/4010acd13d9445b1943eac27cb61aaa2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.682542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b69a68-7948-4c7a-b4a1-0937a110e644 None admin] RBAC: Authorizing `identity:get_project(project_id=4010acd13d9445b1943eac27cb61aaa2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:54.685717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b69a68-7948-4c7a-b4a1-0937a110e644 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:54.689321 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 222/431] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:54 2022] GET /identity/v3/projects/4010acd13d9445b1943eac27cb61aaa2 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:54.704414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d350ee8-5107-4735-947a-580c3ba2a829 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:54.914285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402789f-97c8-4f99-b14e-070c4cf6f40c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.915610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402789f-97c8-4f99-b14e-070c4cf6f40c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.921266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402789f-97c8-4f99-b14e-070c4cf6f40c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.924605 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 223/432] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:54 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:54.934668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f48ae-9e35-43d4-8afd-000e7cfb3da3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:54.935132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f48ae-9e35-43d4-8afd-000e7cfb3da3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:54.935676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f48ae-9e35-43d4-8afd-000e7cfb3da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:54.941344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d350ee8-5107-4735-947a-580c3ba2a829 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:54.962041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:54.970794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-382f48ae-9e35-43d4-8afd-000e7cfb3da3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:54.973866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-382f48ae-9e35-43d4-8afd-000e7cfb3da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:55.022074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d409f9-4ae2-473d-8c85-1ea637c61fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:55.023221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d409f9-4ae2-473d-8c85-1ea637c61fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:55.023793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d409f9-4ae2-473d-8c85-1ea637c61fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:55.024205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d409f9-4ae2-473d-8c85-1ea637c61fb7 None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:55.026918 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d409f9-4ae2-473d-8c85-1ea637c61fb7 None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:55.030900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d409f9-4ae2-473d-8c85-1ea637c61fb7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:55.036526 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 210/433] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:54 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:55.039069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b33f3-03e1-4e3f-90a2-10cc398237d5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:55.039548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b33f3-03e1-4e3f-90a2-10cc398237d5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:55.040016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b33f3-03e1-4e3f-90a2-10cc398237d5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:55.048535 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 211/434] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:44:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:44:55.052171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b33f3-03e1-4e3f-90a2-10cc398237d5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:55.061593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:55.135598 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 224/435] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:44:55.135598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58f64f-a7bb-48bf-ab81-d3169960ebb4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:55.135598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58f64f-a7bb-48bf-ab81-d3169960ebb4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:55.135598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58f64f-a7bb-48bf-ab81-d3169960ebb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:55.159223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0c58f64f-a7bb-48bf-ab81-d3169960ebb4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:44:55.160759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0c58f64f-a7bb-48bf-ab81-d3169960ebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:44:55.256115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b33f3-03e1-4e3f-90a2-10cc398237d5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:55.277352 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 225/436] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:44:55.278779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58f64f-a7bb-48bf-ab81-d3169960ebb4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:55.333685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:55.335206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:55.335985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:55.336769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:55.339860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=4010acd13d9445b1943eac27cb61aaa2) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:55.340810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:55.348407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:55.361561 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 212/437] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:55 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=4010acd13d9445b1943eac27cb61aaa2&role.id=896da438cb834959bd42187c944ea2f7 => generated 611 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:55.508195 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58f64f-a7bb-48bf-ab81-d3169960ebb4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:55.541062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:55.583968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:55.585870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:55.586723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:55.587511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:55.610825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:55.621204 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:55.622734 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 25 00:44:55.624247 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:55.628706 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 226/438] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:55.643556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:55.656624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:55.737861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:55.861253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3fc5e-e304-4d3b-af91-192737549eb1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:55.889680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2177a-1b56-44e9-959d-cd1c44ccbc5a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:55.933638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:55.934678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:55.935158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:55.935635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:55.936976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:55.937528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:55.940193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:55.950337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:55.951527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:55.951989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:55.952572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:55.956725 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 213/439] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:44:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:55.957496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:55.974385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:55.981567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:55.983408 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:44:55.985295 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 227/440] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:44:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:55.992984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:56.029683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:56.183107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b090e7-e4be-4d79-9dd4-e74c194da445 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:56.185638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:56.196679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e399f2c-8895-4049-afc5-64cd44d0ea35 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:56.252152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:56.252943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:56.253227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:56.253504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:56.263355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:56.264457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:56.264926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:56.265308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:56.268299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:44:56.279423 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:44:56.282306 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 214/441] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:44:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:44:56.285186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:56.392304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:56.395073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:56.419207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edf4f5-fc56-4ff4-9793-5123cd930691 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:56.438660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:56.466512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:56.467389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:56.467887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:56.468357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:56.469620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:56.470172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:56.472670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:56.495500 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 215/442] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:56.495500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:56.534302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:56.571315 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 228/443] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:56.573374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:56.688415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da7570-65ef-461f-981d-c5dd0cc4d730 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:56.755530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:56.756521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:56.757033 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:56.757521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:56.758807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:56.759365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:56.762416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:56.784833 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 216/444] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:44:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:56.784833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:56.785250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9b384-3c00-434d-8de4-786d09b73a81 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:56.826037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:56.846532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:56.847753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:56.848331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:56.848888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:56.953479 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:56.984644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae051e-2625-43b7-a15e-1e2ed9da77a0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:56.996277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:56.999150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:57.047749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:57.048447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:57.048738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:57.049011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:57.050290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:57.051812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:57.053822 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 217/445] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:56 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:57.055210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:57.198688 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 229/446] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:57.200466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:57.230136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0190e26-0bd4-4639-96bd-00ad75852e1f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:57.251761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:57.286014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:57.286897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:57.287311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:57.287770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:57.327680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:57.377005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8f83-f237-4a4d-90de-e26fffda2e34 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:57.390860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:57.392988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:57.438392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:57.439702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:57.440198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:57.440723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:57.442737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:57.445608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:57.448358 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 230/447] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:57 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:57.449812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:57.551807 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 218/448] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:44:57.553287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:57.606432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:57.622017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b9cc3-9bd2-45f2-9a08-5ebcc01131ac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:57.647281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:57.685032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:57.685953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:57.686341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:57.686731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] PATH_INFO: `/v3/projects/3708ba2402cc4ee7abd81255a285d2b9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:57.689260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] RBAC: Authorizing `identity:get_project(project_id=3708ba2402cc4ee7abd81255a285d2b9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:57.691457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:57.693721 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 231/449] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:57 2022] GET /identity/v3/projects/3708ba2402cc4ee7abd81255a285d2b9 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:57.694917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:57.734599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eecb2-4cdb-4fe4-ba00-79423ca71b37 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:57.795780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:57.796907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:57.797410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:57.797894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:57.799515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=896da438cb834959bd42187c944ea2f7, scope.project.id=a83742d0de534530bf5beab6ecf7fe70) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:57.800074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:57.802917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:57.824494 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 219/450] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:57 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=896da438cb834959bd42187c944ea2f7 => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:57.824494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:57.890341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575ebf8-87b5-4388-8e3d-eb9d81fb568e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:57.949403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:57.951282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:57.951769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:57.952242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] PATH_INFO: `/v3/projects/a83742d0de534530bf5beab6ecf7fe70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:57.954558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] RBAC: Authorizing `identity:get_project(project_id=a83742d0de534530bf5beab6ecf7fe70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:57.957449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:57.959888 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 232/451] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:44:57 2022] GET /identity/v3/projects/a83742d0de534530bf5beab6ecf7fe70 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:57.967088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:58.028175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:58.041272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917edf7-b45b-4306-99d9-4513cd61632f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:58.075034 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:58.113842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:58.114929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:58.116630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:58.116630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] PATH_INFO: `/v3/groups/fa41eed58c8e4624ad28ba89d255c4b5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:58.118050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] RBAC: Authorizing `identity:get_group(group_id=fa41eed58c8e4624ad28ba89d255c4b5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:58.120881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:58.124438 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 220/452] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:57 2022] GET /identity/v3/groups/fa41eed58c8e4624ad28ba89d255c4b5 => generated 229 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:44:58.135391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:58.181962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765a171-1690-4fc3-8db7-95c55a839100 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:58.250730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:58.251807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:58.252209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:58.252702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] PATH_INFO: `/v3/groups/2555bae83c2141ea992f0bab7d6601de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:58.257009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] RBAC: Authorizing `identity:get_group(group_id=2555bae83c2141ea992f0bab7d6601de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:58.257598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:58.265469 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 233/453] 10.222.0.26 () {56 vars in 1180 bytes} [Sun Sep 25 00:44:57 2022] GET /identity/v3/groups/2555bae83c2141ea992f0bab7d6601de => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:58.269190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:44:58.347424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461bee2-6b6d-4dba-99b3-2128da9ad62b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:58.427458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd0b6f-084e-4fce-aa9c-b3c6abf650f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:58.428573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b6f-084e-4fce-aa9c-b3c6abf650f9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:58.429081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b6f-084e-4fce-aa9c-b3c6abf650f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:58.429576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b6f-084e-4fce-aa9c-b3c6abf650f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:58.431002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0b6f-084e-4fce-aa9c-b3c6abf650f9 None admin] RBAC: Adding query filter params (group.id=fa41eed58c8e4624ad28ba89d255c4b5, role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=3708ba2402cc4ee7abd81255a285d2b9) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:58.432170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0b6f-084e-4fce-aa9c-b3c6abf650f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:58.435051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0b6f-084e-4fce-aa9c-b3c6abf650f9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:58.446953 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 221/454] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:58 2022] GET /identity/v3/role_assignments?group.id=fa41eed58c8e4624ad28ba89d255c4b5&scope.project.id=3708ba2402cc4ee7abd81255a285d2b9&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 611 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:44:58.461311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:44:58.471447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5de23-f5d4-4b2a-ba23-43d5e60a32a9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:44:58.516277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daca03f-7c28-4196-91bd-c126ea91f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:44:58.517001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daca03f-7c28-4196-91bd-c126ea91f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:44:58.517356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daca03f-7c28-4196-91bd-c126ea91f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:44:58.517710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daca03f-7c28-4196-91bd-c126ea91f5d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:44:58.518537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daca03f-7c28-4196-91bd-c126ea91f5d2 None admin] RBAC: Adding query filter params (group.id=2555bae83c2141ea992f0bab7d6601de, role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=a83742d0de534530bf5beab6ecf7fe70) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:44:58.518905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daca03f-7c28-4196-91bd-c126ea91f5d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:44:58.520577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daca03f-7c28-4196-91bd-c126ea91f5d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:44:58.544442 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 234/455] 10.222.0.26 () {56 vars in 1376 bytes} [Sun Sep 25 00:44:58 2022] GET /identity/v3/role_assignments?group.id=2555bae83c2141ea992f0bab7d6601de&scope.project.id=a83742d0de534530bf5beab6ecf7fe70&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:03.590336 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915607f-9194-4e9b-8144-b2a305560cd0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:03.591563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915607f-9194-4e9b-8144-b2a305560cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:03.592034 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915607f-9194-4e9b-8144-b2a305560cd0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:03.595161 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 222/456] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:45:03.603703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecbc8c-207d-4498-9157-4c9d86dac581 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:03.605339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecbc8c-207d-4498-9157-4c9d86dac581 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:03.606139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecbc8c-207d-4498-9157-4c9d86dac581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:03.657310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8cecbc8c-207d-4498-9157-4c9d86dac581 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:03.659831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8cecbc8c-207d-4498-9157-4c9d86dac581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:03.829836 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 235/457] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:45:03.884203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4d97-df0b-45f2-829c-42f10a8c1261 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:03.885850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4d97-df0b-45f2-829c-42f10a8c1261 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:03.886674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4d97-df0b-45f2-829c-42f10a8c1261 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:03.892435 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 223/458] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:03.900821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4b6b8-75fa-4004-91b9-bf313dc52608 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:03.901285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4b6b8-75fa-4004-91b9-bf313dc52608 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:03.901753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4b6b8-75fa-4004-91b9-bf313dc52608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:03.936875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e613654-b853-49f8-a857-102c10e48bda None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:03.937700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e613654-b853-49f8-a857-102c10e48bda None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:03.938358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e613654-b853-49f8-a857-102c10e48bda None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:03.946827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-23c4b6b8-75fa-4004-91b9-bf313dc52608 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:03.946827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23c4b6b8-75fa-4004-91b9-bf313dc52608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:03.948203 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 224/459] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:45:03.950805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cbb11-a3df-4083-95b9-eff2032bcf97 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:03.951614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cbb11-a3df-4083-95b9-eff2032bcf97 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:03.952362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cbb11-a3df-4083-95b9-eff2032bcf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:04.006622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-df9cbb11-a3df-4083-95b9-eff2032bcf97 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:04.009302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-df9cbb11-a3df-4083-95b9-eff2032bcf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:04.109614 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 236/460] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:04.182783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:04.242545 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 225/461] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:04.384276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4b6b8-75fa-4004-91b9-bf313dc52608 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:04.433076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:04.649832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4b6b8-75fa-4004-91b9-bf313dc52608 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:04.683504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cbb11-a3df-4083-95b9-eff2032bcf97 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:04.735862 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f559fb49-85ab-45e3-ad32-036506a0a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:04.737339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559fb49-85ab-45e3-ad32-036506a0a8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:04.738939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559fb49-85ab-45e3-ad32-036506a0a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:04.739668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f559fb49-85ab-45e3-ad32-036506a0a8a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:04.766486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559fb49-85ab-45e3-ad32-036506a0a8a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:04.770111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f559fb49-85ab-45e3-ad32-036506a0a8a3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f559fb49-85ab-45e3-ad32-036506a0a8a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:04.808258 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:04.810383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:04.812340 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:04.812340 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:04.812340 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:04.814839 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 237/462] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:04.821310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1460e-94d1-4549-aaa9-bcc96eb0dc9d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:04.821310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1460e-94d1-4549-aaa9-bcc96eb0dc9d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:04.821310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1460e-94d1-4549-aaa9-bcc96eb0dc9d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:04.825351 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 238/463] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:04.830618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1460e-94d1-4549-aaa9-bcc96eb0dc9d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:04.932130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cbb11-a3df-4083-95b9-eff2032bcf97 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:05.001015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:05.015910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea3fa6-5cce-4898-b684-3e607c114cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:05.017128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3fa6-5cce-4898-b684-3e607c114cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.017639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3fa6-5cce-4898-b684-3e607c114cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.018161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea3fa6-5cce-4898-b684-3e607c114cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.041951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea3fa6-5cce-4898-b684-3e607c114cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:05.044986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea3fa6-5cce-4898-b684-3e607c114cf8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:05.072484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1460e-94d1-4549-aaa9-bcc96eb0dc9d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-26ea3fa6-5cce-4898-b684-3e607c114cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:05.076683 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:05.078469 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:05.080167 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:05.080167 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:05.080167 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:05.080167 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 226/464] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:05.081684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367babf-ea09-4ace-8c34-3fbe1149d4be None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.082167 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367babf-ea09-4ace-8c34-3fbe1149d4be None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.082640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367babf-ea09-4ace-8c34-3fbe1149d4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.114748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2367babf-ea09-4ace-8c34-3fbe1149d4be None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:05.116953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2367babf-ea09-4ace-8c34-3fbe1149d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:05.150550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:05.151789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.152288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.152875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.154234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:05.154785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:05.157613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:05.170829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:05.180058 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 239/465] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:05.182591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:05.242618 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 227/466] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:05.244812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367babf-ea09-4ace-8c34-3fbe1149d4be None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:05.417815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c03c6a-e4e3-404c-974b-c87fdd9a24cf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:05.459729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367babf-ea09-4ace-8c34-3fbe1149d4be None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:05.484108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:05.495702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:05.498015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.498840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.499782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.524412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:05.527297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:05.536152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:05.537896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.538735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.539432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.541498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:05.542329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:05.546430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:05.560219 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:05.561900 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:05.563278 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:05.563278 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:05.563278 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:05.563278 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 240/467] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:05.563852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:05.594198 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 228/468] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:05.597091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:05.657153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:05.784131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212325-ae8f-4b55-86bc-01e95e503dbf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:05.834155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1c5a6-db05-4b00-8a6a-6eea9d122f36 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:05.852537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:05.853714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.854238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.854750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.856035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:05.858947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:05.896949 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 241/469] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:45:05 2022] POST /identity/v3/users => generated 246 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:05.901910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff547c-3233-4e79-b391-050e2a4baeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:05.903063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff547c-3233-4e79-b391-050e2a4baeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.903528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff547c-3233-4e79-b391-050e2a4baeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.904013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff547c-3233-4e79-b391-050e2a4baeb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.905414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff547c-3233-4e79-b391-050e2a4baeb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:05.905976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff547c-3233-4e79-b391-050e2a4baeb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:05.906238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:05.909089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff547c-3233-4e79-b391-050e2a4baeb5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:05.934185 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 229/470] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:05.936618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4370-a5c8-438d-9e65-4911e4b61b4b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:05.937101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4370-a5c8-438d-9e65-4911e4b61b4b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:05.937580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4370-a5c8-438d-9e65-4911e4b61b4b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:05.945943 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 230/471] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:05.945943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4370-a5c8-438d-9e65-4911e4b61b4b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:06.003749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:06.129742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:06.136566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66490e43-b5b4-48a8-b40f-9ac7bebf5b18 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:06.172051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4370-a5c8-438d-9e65-4911e4b61b4b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:06.213598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:06.214803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:06.215328 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:06.215803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:06.217037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:06.219737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:06.250594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d73ead-70f7-4889-aad9-8d8e1ffbb6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:06.251781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d73ead-70f7-4889-aad9-8d8e1ffbb6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:06.252263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d73ead-70f7-4889-aad9-8d8e1ffbb6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:06.252799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d73ead-70f7-4889-aad9-8d8e1ffbb6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:06.261323 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 242/472] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:45:05 2022] POST /identity/v3/users => generated 248 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:06.263376 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:06.279349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d73ead-70f7-4889-aad9-8d8e1ffbb6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:06.282251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d73ead-70f7-4889-aad9-8d8e1ffbb6f4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e4d73ead-70f7-4889-aad9-8d8e1ffbb6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:06.317530 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:06.319362 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:06.321071 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:06.321071 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:06.321071 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:06.321071 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 231/473] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:06.323050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b562f-50b2-47f0-8caf-10d8e19bd2ec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:06.323871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b562f-50b2-47f0-8caf-10d8e19bd2ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:06.329974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b562f-50b2-47f0-8caf-10d8e19bd2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:06.373689 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8a3b562f-50b2-47f0-8caf-10d8e19bd2ec None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:06.375257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8a3b562f-50b2-47f0-8caf-10d8e19bd2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:06.519703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:06.524254 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 232/474] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:06.526462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b562f-50b2-47f0-8caf-10d8e19bd2ec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:06.536028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7165-c3a2-4eb5-b1de-0f5f40068c99 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:06.611028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:06.612083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:06.612612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:06.613150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:06.614497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:06.617423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:06.652539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:06.683945 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 243/475] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:45:06 2022] POST /identity/v3/users => generated 248 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:06.738838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b562f-50b2-47f0-8caf-10d8e19bd2ec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:06.819399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:06.820904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:06.821699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:06.822949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:06.824518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:06.825444 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:06.828599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:06.864650 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 233/476] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:06.874382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:07.095570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1085-fc55-4ca8-9d52-eba7254f299a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:07.177548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:07.178331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:07.179502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:07.179909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:07.180443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:07.181813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:07.184755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:07.234675 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 244/477] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:45:06 2022] POST /identity/v3/users => generated 249 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:07.458403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe026fe-2d21-4b11-a3db-dac054c0ab07 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:07.526091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:07.553790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:07.554871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:07.555365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:07.555861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:07.557242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:07.559930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:07.582010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 25 00:45:07.582010 dv-cinder-master-3-smb devstack@keystone.service[81500]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 25 00:45:07.582010 dv-cinder-master-3-smb devstack@keystone.service[81500]: [parameters: {'id': 'e8210b5d83324a10a39742d85618fb1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 25 00:45:07.582010 dv-cinder-master-3-smb devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 25 00:45:07.624027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return ref.to_dict() Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 25 00:45:07.625809 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 25 00:45:07.627590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 25 00:45:07.631010 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 25 00:45:07.634983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 25 00:45:07.638577 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'id': 'e8210b5d83324a10a39742d85618fb1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:07.648171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 25 00:45:07.651528 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:07.656406 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 234/478] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Sep 25 00:45:07 2022] POST /identity/v3/roles => generated 144 bytes in 450 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:07.656655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:07.683538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:07.902818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d88d65-e770-48be-aff7-238dbd14459b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:07.980360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e93850-f554-4d1c-9394-c3f68fce833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:07.982045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93850-f554-4d1c-9394-c3f68fce833b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:07.982829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93850-f554-4d1c-9394-c3f68fce833b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:07.983580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e93850-f554-4d1c-9394-c3f68fce833b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:08.011908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e93850-f554-4d1c-9394-c3f68fce833b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-99e93850-f554-4d1c-9394-c3f68fce833b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:08.023545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:08.026092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 25 00:45:08.029098 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:08.034183 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 245/479] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:45:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:08.046088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:08.332334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf8e52-cf02-4565-a3e5-defe124a8e6c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:08.445824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb9cee-e3a9-4b30-b572-0a7ce5e3645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:08.447670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9cee-e3a9-4b30-b572-0a7ce5e3645c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:08.448161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9cee-e3a9-4b30-b572-0a7ce5e3645c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:08.448678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb9cee-e3a9-4b30-b572-0a7ce5e3645c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:08.449949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb9cee-e3a9-4b30-b572-0a7ce5e3645c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:08.450488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb9cee-e3a9-4b30-b572-0a7ce5e3645c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:08.453582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb9cee-e3a9-4b30-b572-0a7ce5e3645c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:08.479084 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 235/480] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:10.638379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d232-aac1-490f-a33e-13caf721c1aa None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:10.639188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d232-aac1-490f-a33e-13caf721c1aa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:10.639188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d232-aac1-490f-a33e-13caf721c1aa None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:10.642299 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 246/481] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:10.651666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133e17f-809a-48e6-8f8d-4cd74d14fc80 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:10.652459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133e17f-809a-48e6-8f8d-4cd74d14fc80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:10.652783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133e17f-809a-48e6-8f8d-4cd74d14fc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:10.694357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6133e17f-809a-48e6-8f8d-4cd74d14fc80 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:10.696346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6133e17f-809a-48e6-8f8d-4cd74d14fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:10.809649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:10.872240 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 236/482] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:11.553333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507d232-aac1-490f-a33e-13caf721c1aa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:11.563732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a36f9-4133-4296-877a-04089c1b8e80 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:11.564171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a36f9-4133-4296-877a-04089c1b8e80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:11.564650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a36f9-4133-4296-877a-04089c1b8e80 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:11.570235 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 237/483] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:11.572728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ac2a-7983-4316-8b59-a7ea3fd08168 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:11.573188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ac2a-7983-4316-8b59-a7ea3fd08168 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:11.573680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32ac2a-7983-4316-8b59-a7ea3fd08168 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:11.578858 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 238/484] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:11.586897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42617-435d-480b-bb0a-fbbf8762da56 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:11.587182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42617-435d-480b-bb0a-fbbf8762da56 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:11.588295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f42617-435d-480b-bb0a-fbbf8762da56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:11.620235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d7f42617-435d-480b-bb0a-fbbf8762da56 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:11.622646 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d7f42617-435d-480b-bb0a-fbbf8762da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:11.719472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:11.773696 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 239/485] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:11.777483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f6822-257d-4cc4-84df-a573c8bc9699 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:11.778315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f6822-257d-4cc4-84df-a573c8bc9699 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:11.779084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f6822-257d-4cc4-84df-a573c8bc9699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:11.810199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507d232-aac1-490f-a33e-13caf721c1aa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:11.830909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f23f6822-257d-4cc4-84df-a573c8bc9699 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:11.838000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f23f6822-257d-4cc4-84df-a573c8bc9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:11.913310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:11.914460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:11.914970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:11.915470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:11.937167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:11.948239 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:11.949818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:11.951154 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 247/486] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:11.960031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:12.035558 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 240/487] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:12.202064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153a2a-6391-43e8-8fb1-909021008849 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:12.281103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:12.299060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:12.299965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:12.300397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:12.300860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:12.302082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:12.302626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:12.313620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:12.331857 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 248/488] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:12.341452 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f6822-257d-4cc4-84df-a573c8bc9699 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:12.371244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:12.437038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:12.547253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f6822-257d-4cc4-84df-a573c8bc9699 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:12.591605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb0519-38d4-4817-b48d-972593999db7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:12.600579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d2f51-b4d9-4dbd-86c1-638f173c501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:12.601507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d2f51-b4d9-4dbd-86c1-638f173c501a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:12.601803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d2f51-b4d9-4dbd-86c1-638f173c501a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:12.602129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d2f51-b4d9-4dbd-86c1-638f173c501a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:12.619021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d2f51-b4d9-4dbd-86c1-638f173c501a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:12.621100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d2f51-b4d9-4dbd-86c1-638f173c501a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0f5d2f51-b4d9-4dbd-86c1-638f173c501a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:12.642637 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:12.644460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:12.646122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:12.646122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:12.646122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:12.646122 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 241/489] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:12.646122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36da592-039e-4ba2-9ac9-217d5caf773c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:12.647704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36da592-039e-4ba2-9ac9-217d5caf773c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:12.647704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36da592-039e-4ba2-9ac9-217d5caf773c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:12.653833 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 242/490] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:12.653833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36da592-039e-4ba2-9ac9-217d5caf773c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:12.677606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:12.678722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:12.679203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:12.679685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:12.702278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:12.712206 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:12.714214 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:12.715855 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:12.720631 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 249/491] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:12.722378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:12.821135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:12.852616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:12.874046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36da592-039e-4ba2-9ac9-217d5caf773c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:12.923384 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843c5f-d001-4603-8c5c-c570607dc013 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:12.957231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e8915-8a98-49df-a97e-3fe8fa3e4702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:12.958371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e8915-8a98-49df-a97e-3fe8fa3e4702 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:12.958854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e8915-8a98-49df-a97e-3fe8fa3e4702 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:12.959362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e8915-8a98-49df-a97e-3fe8fa3e4702 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:12.981824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e8915-8a98-49df-a97e-3fe8fa3e4702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:12.990609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:12.991563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:12.991983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8a5e8915-8a98-49df-a97e-3fe8fa3e4702 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:12.992150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:12.994299 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:12.995854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:12.998615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:12.998615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:12.998615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:12.998615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:13.003196 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 243/492] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:13.005261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42644255-5b76-4481-93c1-b137b8d6b029 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.005759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42644255-5b76-4481-93c1-b137b8d6b029 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.006234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42644255-5b76-4481-93c1-b137b8d6b029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.029580 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 250/493] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:13.029580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:13.040841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-42644255-5b76-4481-93c1-b137b8d6b029 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:13.043099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42644255-5b76-4481-93c1-b137b8d6b029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:13.178808 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 244/494] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:13.180860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42644255-5b76-4481-93c1-b137b8d6b029 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:13.260881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7be5d-f856-409a-9265-18c96c4268e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:13.287502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:13.348776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:13.349887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.350400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.350891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.352523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:13.352909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:13.356244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:13.379799 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 251/495] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:13.379799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:13.385749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:13.426378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42644255-5b76-4481-93c1-b137b8d6b029 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:13.500438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72a0f4-c19e-4e78-b023-2116300fec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:13.501522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72a0f4-c19e-4e78-b023-2116300fec9f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.502019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72a0f4-c19e-4e78-b023-2116300fec9f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.502500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72a0f4-c19e-4e78-b023-2116300fec9f None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.503769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72a0f4-c19e-4e78-b023-2116300fec9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:13.504302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72a0f4-c19e-4e78-b023-2116300fec9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:13.507098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72a0f4-c19e-4e78-b023-2116300fec9f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:13.528429 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 245/496] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:13.528429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39a81f-2064-4c00-9c94-a9faacd3fcca None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.528429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39a81f-2064-4c00-9c94-a9faacd3fcca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.528429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39a81f-2064-4c00-9c94-a9faacd3fcca None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.529879 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 246/497] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:13.537971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39a81f-2064-4c00-9c94-a9faacd3fcca None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:13.584682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa599a-8817-43a2-a6c6-f4c9c5b34e48 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:13.671361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:13.672900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.673607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.674358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.688588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:13.690458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:13.713241 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:13.715294 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 252/498] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:45:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:13.717538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:13.741779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39a81f-2064-4c00-9c94-a9faacd3fcca None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:13.765043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:13.807237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86eded-291d-40d3-925e-fc280c8a168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:13.807899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86eded-291d-40d3-925e-fc280c8a168c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.808353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86eded-291d-40d3-925e-fc280c8a168c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.808861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86eded-291d-40d3-925e-fc280c8a168c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.826908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86eded-291d-40d3-925e-fc280c8a168c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:13.828992 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86eded-291d-40d3-925e-fc280c8a168c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:13.832826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-db86eded-291d-40d3-925e-fc280c8a168c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:13.851050 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:13.852297 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:13.854515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:13.854515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:13.854515 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:13.854515 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 247/499] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:13.854515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e5b7-9b01-4af4-bdb4-cd8f2b4c0cf6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.854515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e5b7-9b01-4af4-bdb4-cd8f2b4c0cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.854515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e5b7-9b01-4af4-bdb4-cd8f2b4c0cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.886157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0dcbd-c8ce-4a32-b7fc-1eb36beb2443 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:13.886568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4646e5b7-9b01-4af4-bdb4-cd8f2b4c0cf6 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:13.887147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4646e5b7-9b01-4af4-bdb4-cd8f2b4c0cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:13.954869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:13.957917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:13.958413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:13.958899 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:13.977983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:13.980406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:14.002760 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:14.004682 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:14.007145 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:14.007145 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:14.007145 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:14.007145 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 253/500] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:14.007145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:14.020095 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 248/501] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:14.020095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646e5b7-9b01-4af4-bdb4-cd8f2b4c0cf6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:14.172884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:14.207811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7c993-fa2a-470f-9e2b-d030d52b2c69 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:14.243218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646e5b7-9b01-4af4-bdb4-cd8f2b4c0cf6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:14.266041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:14.267004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:14.267484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:14.267948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:14.286791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:14.288012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:14.297684 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:14.299393 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:14.300955 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 254/502] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:14.303780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:14.312530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:14.313524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:14.314279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:14.314667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:14.316252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:14.316825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:14.319659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:14.400162 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 249/503] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:45:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:14.401682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:14.497428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf3cb8-8e89-4f72-beec-493563b9dd95 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:14.553536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:14.554431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:14.554928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:14.555382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:14.556932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:14.557279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:14.559310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:14.584632 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 255/504] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:14.584632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:14.587157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:14.595910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662140c-68c7-4dc4-bc8e-7f2af3047b18 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:14.662239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:14.663249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:14.663784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:14.664329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:14.665854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:14.666407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:14.668896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:14.686866 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 250/505] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:14.688268 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:14.736795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbf748-dc3a-40a3-a5dc-27de8ac055f0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:14.763786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:14.796274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:14.797303 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:14.797753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:14.798193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:14.799348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:14.799836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:14.802211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:14.817255 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 256/506] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:14.818429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:14.870749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd227c61-9674-4b82-9d31-d7a7e1a285ef None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:14.929679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:14.935108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:14.936081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:14.936563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:14.937004 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:14.993408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d4032-de43-4858-b456-ce87e571d58a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:15.052099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:15.054212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:15.060472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:15.061868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:15.063495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:15.063495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:15.082949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:15.085760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:15.113383 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:15.115428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:15.117598 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:15.117598 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:15.117598 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:15.117598 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 257/507] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:15.117598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:15.135591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:15.247791 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 251/508] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:15.249062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:15.279503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0825a21-43de-4cf6-a4dc-7fce76c1b77c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:15.310280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:15.343727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:15.344603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:15.345639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:15.346088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:15.358511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:15.360317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:15.377195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:15.379027 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 258/509] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:15.380859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:15.418066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7bf34-6aca-448a-ab4c-8952fbac2997 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:15.472884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:15.473997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:15.474493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:15.474924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:15.487002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:15.489614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:15.515564 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:15.516661 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:15.517823 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 252/510] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:15.519165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:15.528422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:15.568011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c3d3f-632b-4021-8019-fa7543cb7dd7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:15.637041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:15.638324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:15.638841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:15.639478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:15.658062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:15.660680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:15.685747 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:15.687801 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:15.689632 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:15.689632 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:15.689632 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:15.689632 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 259/511] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:15.689632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:15.698484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf37dce-a62b-466d-b165-22cd0620891e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:15.751117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:15.765108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:15.766103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:15.767441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:15.767441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:15.769110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:15.769935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:15.772217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:15.798228 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 253/512] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:15.798228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:15.891856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:15.905563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7f481-c627-4f56-b8ad-e064459ea7f6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:15.968628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2e92b-dd64-46b7-b107-5a8c003451a1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:15.976445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:15.977421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:15.977919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:15.978937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:15.997203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:16.000223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:16.028011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:16.028757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:16.029129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:16.029466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:16.029834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:16.031536 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:16.033255 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:16.033255 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:16.033255 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:16.033573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:16.033573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:16.033820 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 260/513] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:16.034460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:16.034642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:16.059850 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 254/514] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Sep 25 00:45:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:16.061531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:16.188117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:16.227215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cd266-400e-4864-b1c9-d1efc23d8896 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:16.237005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67b88d-61b2-4fe0-a62d-6248bc74457b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:16.276972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:16.296098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:16.296546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:16.297015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:16.297396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:16.297569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:16.297830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:16.298010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:16.298264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:16.298897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:16.299293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:16.299598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:16.300137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:16.301131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:16.302790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:16.321751 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 255/515] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:16.323790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:16.383250 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 261/516] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Sep 25 00:45:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:16.384797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:16.489005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c387c-83fb-4dd0-9759-02f27d50a1d7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:16.550011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:16.550972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:16.551406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:16.551833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:16.553012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:16.555509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:16.556856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6017f-e34c-4d53-9539-1b9b11b07fb0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:16.592740 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 256/517] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:45:16 2022] POST /identity/v3/users => generated 247 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:16.594687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:16.633430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:16.634511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:16.634994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:16.635489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:16.636521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:16.637075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:16.640099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:16.662182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:16.676673 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 262/518] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:16.678283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:16.692055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:16.838781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829cdf9-843c-40bc-ba83-64a07c004d63 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:16.910230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:16.911415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934c4b7-c290-4945-bacb-2d46500fe8cf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:16.913168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:16.913849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:16.914258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:16.984963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:16.985938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:16.986360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:16.986790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:17.003217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:17.006019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:17.029613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:17.031467 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 25 00:45:17.034119 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:17.039075 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 263/519] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:17.040351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:17.058523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:17.061815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:17.145750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:17.164294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:17.259862 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710cbf2-d826-430e-8415-baa8959256cb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:17.286178 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 257/520] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:17.286178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:17.319356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:17.320312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:17.320767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:17.321161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:17.433284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:17.434738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:17.461962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8c59f-9067-46b9-8578-064594aa654c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:17.506743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:17.507558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:17.507967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:17.508356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:17.517837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:17.519588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:17.521898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:17.536447 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:17.538034 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:17.539798 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:17.541926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:17.544459 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 258/521] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:17.545984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:17.625441 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 264/522] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:17.627486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:17.714381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d01c64-a872-4305-a457-4e9308f23a2c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:17.767001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:17.768061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:17.768600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:17.769084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:17.770490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:17.771041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:17.773850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:17.805578 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 259/523] 10.222.0.26 () {56 vars in 1139 bytes} [Sun Sep 25 00:45:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:17.808098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:17.818415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f25357-98c9-49ca-8806-e515c56ed8f1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:17.891348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b265-63f1-4091-be58-e74194db285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:17.892316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:17.892498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b265-63f1-4091-be58-e74194db285f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:17.892896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b265-63f1-4091-be58-e74194db285f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:17.893869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236b265-63f1-4091-be58-e74194db285f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:17.913443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b265-63f1-4091-be58-e74194db285f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:17.918889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236b265-63f1-4091-be58-e74194db285f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b236b265-63f1-4091-be58-e74194db285f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:17.943586 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:17.945510 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:17.949445 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:17.949445 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:17.949445 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:17.949445 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 265/524] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:17.949445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b265-63f1-4091-be58-e74194db285f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:17.972874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:17.975643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a044110-23e1-400b-9955-b4c936716822 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:18.049835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:18.051045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:18.051744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:18.052009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:18.053331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:18.053886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:18.057113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:18.093300 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 260/525] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:18.095130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:18.151664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236b265-63f1-4091-be58-e74194db285f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:18.219710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:18.220805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:18.221305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:18.221779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:18.242365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:18.245794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:18.273335 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:18.275721 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:18.278924 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:18.278924 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:18.278924 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:18.278924 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 266/526] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:18.278924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:18.321555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55e117-986e-485f-aeed-95ae6b3ee16f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:18.364420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:18.394069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:18.395153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:18.395556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:18.395946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:18.406602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:18.485068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e655d2-4784-49ff-a1ce-92b1dc90188e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:18.535251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:18.538127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:18.555627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:18.556908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:18.557553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:18.558222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:18.559602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:18.560131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:18.563010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:18.590119 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 267/527] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:18.590119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:18.745235 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 261/528] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:18.747271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:18.777195 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b495f-59ec-4011-8a62-9c2a169e9833 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:18.783901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:18.822987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:18.843695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:18.845092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:18.845612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:18.846114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:18.847587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=9d8bf513dc544632a09a2d10988a888f) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:18.848153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:18.851150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:18.862878 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 268/529] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:18 2022] GET /identity/v3/role_assignments?user.id=9d8bf513dc544632a09a2d10988a888f&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:18.864183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:18.979555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74dcaf-9908-4c68-afa3-25be8eca128f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:19.068893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:19.070536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:19.071454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:19.072331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:19.074059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:19.074995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:19.077942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:19.106691 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 262/530] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:19.108769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:19.131245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07297c8a-346e-418b-86b7-f01155036cd0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:19.201564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:19.203197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:19.204970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:19.205499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:19.219183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:19.221380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:19.242761 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:19.244784 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:19.245683 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:19.249695 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 269/531] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:19.251601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:19.324210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:19.357804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:19.390035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade1612-2e50-4e63-a21a-5cec95df011d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:19.462360 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:19.463698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:19.464476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:19.465013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:19.466572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b9b5f-1931-4eb8-a3c3-84aea10a229e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:19.492659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:19.495975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:19.523832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:19.525242 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:19.526500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:19.526500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:19.526500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:19.527926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:19.529001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:19.529527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:19.529754 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 263/532] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:19.529959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:19.531331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:19.554552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:19.558793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:19.591827 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:19.593296 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:19.594694 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:19.601989 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 270/533] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:19.603930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:19.734972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b02052-038e-49a7-ac98-0b7492827c9a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:19.802256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:19.821174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:19.823002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:19.823922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:19.824820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:19.826584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:19.827506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:19.830713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:19.847964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd30a73-7a41-475d-b196-81b26fce159c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:19.858456 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 264/534] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:19.860719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:19.876170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:19.927148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:19.928278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:19.928709 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:19.929220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:19.930495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:19.931011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:19.934326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:19.966285 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 271/535] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:19.967715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:20.076229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4900a-c4ce-403b-ab50-bfa67ba16cd1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:20.145752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d931ac2-f7e3-424c-b75c-5040039fbe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:20.146591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931ac2-f7e3-424c-b75c-5040039fbe29 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:20.147030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931ac2-f7e3-424c-b75c-5040039fbe29 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:20.147411 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931ac2-f7e3-424c-b75c-5040039fbe29 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:20.148662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d931ac2-f7e3-424c-b75c-5040039fbe29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:20.149128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d931ac2-f7e3-424c-b75c-5040039fbe29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:20.151774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d931ac2-f7e3-424c-b75c-5040039fbe29 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:20.180448 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 265/536] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:20.180448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64185b-5574-4c4f-9e42-add39b115858 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:20.180448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64185b-5574-4c4f-9e42-add39b115858 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:20.180448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64185b-5574-4c4f-9e42-add39b115858 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:20.185712 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 266/537] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:20.187313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea23b0-111f-4961-8f55-392adbb0f7bf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:20.190286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64185b-5574-4c4f-9e42-add39b115858 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:20.253405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:20.254351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:20.254808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:20.255268 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:20.256624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=009181b0acc24632a5614482cdebb970) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:20.257118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:20.260276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:20.270511 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 272/538] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:19 2022] GET /identity/v3/role_assignments?user.id=009181b0acc24632a5614482cdebb970&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:20.276885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:20.290153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:20.365689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:20.396616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64185b-5574-4c4f-9e42-add39b115858 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:20.467589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572016a-a969-4444-a7c0-e94e16228bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:20.468983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572016a-a969-4444-a7c0-e94e16228bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:20.469536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572016a-a969-4444-a7c0-e94e16228bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:20.470068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572016a-a969-4444-a7c0-e94e16228bf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:20.482515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572016a-a969-4444-a7c0-e94e16228bf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:20.485404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572016a-a969-4444-a7c0-e94e16228bf9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d572016a-a969-4444-a7c0-e94e16228bf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:20.503540 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:20.505536 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:20.508016 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:20.510033 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 267/539] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:20.512780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317760c-207d-4f71-afa3-0560b10b5aa5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:20.513400 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317760c-207d-4f71-afa3-0560b10b5aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:20.514576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317760c-207d-4f71-afa3-0560b10b5aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:20.542160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c5f26-8b76-47c9-80de-0ad92fbce503 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:20.548618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7317760c-207d-4f71-afa3-0560b10b5aa5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:20.549945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7317760c-207d-4f71-afa3-0560b10b5aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:20.626240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:20.627903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:20.628676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:20.629599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:20.631057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=bddb3327380d43ef9b24277a3b308b0e) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:20.631776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:20.634644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:20.653025 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 273/540] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:20 2022] GET /identity/v3/role_assignments?user.id=bddb3327380d43ef9b24277a3b308b0e&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 259 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:20.654586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:20.707398 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 268/541] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:20.913823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:20.929630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9cbd5-7a8f-4d26-9157-e28b00e6e970 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:21.001217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:21.002282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:21.002789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:21.003275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:21.004636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:21.005196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:21.008028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:21.048437 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 274/542] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:21.058665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317760c-207d-4f71-afa3-0560b10b5aa5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:21.160935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:21.290457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317760c-207d-4f71-afa3-0560b10b5aa5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:21.357849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:21.359279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:21.359978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:21.360685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:21.362410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=8427b2d4e8c040d0a1b00e25e3b58e30) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:21.363384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:21.366490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:21.378459 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 269/543] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:21 2022] GET /identity/v3/role_assignments?user.id=8427b2d4e8c040d0a1b00e25e3b58e30&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:21.570587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:21.838111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8937b-adfe-459a-89dd-6c19d847d065 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:21.938300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:21.939395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:21.939886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:21.940354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:21.973684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:21.983439 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:21.985277 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:21.987019 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:21.992584 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 275/544] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:21.998782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:22.261012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163f05b-c0e8-459a-952b-f0622a7f5cc1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:22.268963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:22.337792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091f4b7-d6d3-4d40-b341-f1a8e2589eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:22.338871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091f4b7-d6d3-4d40-b341-f1a8e2589eeb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:22.339173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091f4b7-d6d3-4d40-b341-f1a8e2589eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:22.339453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091f4b7-d6d3-4d40-b341-f1a8e2589eeb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:22.340658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091f4b7-d6d3-4d40-b341-f1a8e2589eeb None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:22.341219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091f4b7-d6d3-4d40-b341-f1a8e2589eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:22.343905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091f4b7-d6d3-4d40-b341-f1a8e2589eeb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:22.361150 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 270/545] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:22.370253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:22.435638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:22.666656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec111832-9eab-4205-9529-6352ce41f10b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:22.758491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:22.758940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab3b4-6219-493e-bd1e-d3a3fff561ab None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:22.759494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab3b4-6219-493e-bd1e-d3a3fff561ab None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:22.759660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:22.760437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab3b4-6219-493e-bd1e-d3a3fff561ab None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:22.760674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:22.760674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:22.762233 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 271/546] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:22.774247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e17e-7522-4641-96dc-6201ca993a72 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:22.774685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e17e-7522-4641-96dc-6201ca993a72 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:22.775653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5e17e-7522-4641-96dc-6201ca993a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:22.785714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:22.788617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:22.819434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:22.823017 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:22.825309 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:22.825309 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:22.825309 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:22.825649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b0f5e17e-7522-4641-96dc-6201ca993a72 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:22.825649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b0f5e17e-7522-4641-96dc-6201ca993a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:22.826057 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 276/547] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:22.836880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:23.077412 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 272/548] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:23.092972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:23.162254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fe96f-078a-4afb-bc61-152d494d4108 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:23.250289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:23.251146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:23.251645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:23.252042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:23.252849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:23.253183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:23.255088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:23.279143 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 277/549] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:23.288061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e17e-7522-4641-96dc-6201ca993a72 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:23.432088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:23.444627 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5e17e-7522-4641-96dc-6201ca993a72 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:23.513965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:23.514854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:23.515239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:23.515524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:23.533883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:23.535653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:23.567081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:23.569096 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:23.571827 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 273/550] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:45:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:23.581970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:23.615065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:23.816684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41085a-34cf-473b-8470-2eb565e4df4a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:23.865467 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:23.889426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:23.897114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:23.898112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:23.898878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:23.900782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:23.901759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:23.904941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:23.933389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140b059-4198-4f7f-9d3e-3279c256ea48 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:23.984589 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 278/551] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Sep 25 00:45:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:23.990724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:24.022588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:24.023902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:24.024425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:24.025035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:24.069076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:24.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:24.094886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:24.096588 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:24.099635 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 274/552] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 485 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:24.106108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:24.176493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:24.364900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3046fb7-a0db-4acd-9365-51c0acbd2ca1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:24.434475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e612320-389c-42b7-a979-2fc79944bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:24.435626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e612320-389c-42b7-a979-2fc79944bc82 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:24.436227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e612320-389c-42b7-a979-2fc79944bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:24.436681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e612320-389c-42b7-a979-2fc79944bc82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:24.459909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2c5ac-a978-4f8d-af7f-60ae4a88573e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:24.548938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:24.549972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:24.550468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:24.551037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:24.552308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:24.552889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:24.555641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:24.574303 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 275/553] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:24.583155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:24.618719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:24.650026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e612320-389c-42b7-a979-2fc79944bc82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:24.651761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e612320-389c-42b7-a979-2fc79944bc82 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:24.739690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:24.789836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83116819-e5d3-44ea-80ac-f692463236b2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:24.849550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:24.850282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:24.850588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:24.850998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:24.865815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:24.867473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:24.881435 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 279/554] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 892 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 25 00:45:24.885494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d6c23-9b47-4fc0-a91f-e0ba960f7da3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:24.886328 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:24.888917 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:24.890739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:24.890739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:24.890739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:24.891828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d6c23-9b47-4fc0-a91f-e0ba960f7da3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:24.892984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d6c23-9b47-4fc0-a91f-e0ba960f7da3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:24.893333 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 276/555] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:24.894489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:24.896597 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 280/556] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:24 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:24.903060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d6c23-9b47-4fc0-a91f-e0ba960f7da3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:25.039567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a400b5-d2ce-4d1e-aad2-4b8199eb53cb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:25.085215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:25.098241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12004b-29b4-4693-bada-e68fcf790818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:25.099409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12004b-29b4-4693-bada-e68fcf790818 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.099889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12004b-29b4-4693-bada-e68fcf790818 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.100413 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12004b-29b4-4693-bada-e68fcf790818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.121825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:25.124782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12004b-29b4-4693-bada-e68fcf790818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:25.129114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12004b-29b4-4693-bada-e68fcf790818 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4f12004b-29b4-4693-bada-e68fcf790818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:25.153865 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:25.156285 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:25.158277 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:25.158277 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:25.158277 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:25.158557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d6c23-9b47-4fc0-a91f-e0ba960f7da3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:25.160848 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 277/557] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:25.163422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c264-9c07-41bb-8c30-1b40f3c34f09 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.165035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c264-9c07-41bb-8c30-1b40f3c34f09 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.168867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c264-9c07-41bb-8c30-1b40f3c34f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.199294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fbe1c264-9c07-41bb-8c30-1b40f3c34f09 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:25.201122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fbe1c264-9c07-41bb-8c30-1b40f3c34f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:25.232065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:25.233370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.234052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.234656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.236008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:25.236774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:25.239673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:25.268762 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 281/558] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:25.268762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:25.319714 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 278/559] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:25.321353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1c264-9c07-41bb-8c30-1b40f3c34f09 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:25.477461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d5605-000b-4795-a49a-5d0dba6e5d0b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:25.551751 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc554c4-cc84-45a8-acee-7f6bfb393057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:25.552793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc554c4-cc84-45a8-acee-7f6bfb393057 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.553130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc554c4-cc84-45a8-acee-7f6bfb393057 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.553415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc554c4-cc84-45a8-acee-7f6bfb393057 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.554106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc554c4-cc84-45a8-acee-7f6bfb393057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:25.554489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc554c4-cc84-45a8-acee-7f6bfb393057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:25.555952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc554c4-cc84-45a8-acee-7f6bfb393057 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:25.560369 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1c264-9c07-41bb-8c30-1b40f3c34f09 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:25.578598 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 282/560] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:25.581375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f2f60-272e-4c3d-95ac-c7b73a8335eb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.582219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f2f60-272e-4c3d-95ac-c7b73a8335eb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.583078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f2f60-272e-4c3d-95ac-c7b73a8335eb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.586177 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 283/561] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:25.590029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f2f60-272e-4c3d-95ac-c7b73a8335eb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:25.592572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:25.611047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:25.652426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f19b7-8c26-4a82-bfb8-6a88522dbd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:25.653651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f19b7-8c26-4a82-bfb8-6a88522dbd7d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.654231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f19b7-8c26-4a82-bfb8-6a88522dbd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.654772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f19b7-8c26-4a82-bfb8-6a88522dbd7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.664309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f19b7-8c26-4a82-bfb8-6a88522dbd7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:25.666877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f19b7-8c26-4a82-bfb8-6a88522dbd7d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c15f19b7-8c26-4a82-bfb8-6a88522dbd7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:25.681019 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:25.682189 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:25.683281 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:25.688488 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 279/562] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:45:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:25.690931 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b7a2f-2213-4158-93a6-985fb620df76 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.691661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b7a2f-2213-4158-93a6-985fb620df76 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.692606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b7a2f-2213-4158-93a6-985fb620df76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.725213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e68b7a2f-2213-4158-93a6-985fb620df76 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:25.727826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e68b7a2f-2213-4158-93a6-985fb620df76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:25.791318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f2f60-272e-4c3d-95ac-c7b73a8335eb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:25.860422 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 280/563] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:25.860422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b7a2f-2213-4158-93a6-985fb620df76 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:25.860865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:25.860865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:25.860865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:25.860865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:25.865379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:25.867947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:25.883415 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:25.886656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 284/564] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:25.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:25.983808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:26.034579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f4219-8279-4f6e-933a-3e3765c62134 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:26.041948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:26.047667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b7a2f-2213-4158-93a6-985fb620df76 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:26.100400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e921-a44b-40bf-8b3f-73cb5f4c0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:26.101491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e921-a44b-40bf-8b3f-73cb5f4c0633 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.102092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e921-a44b-40bf-8b3f-73cb5f4c0633 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.102582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e921-a44b-40bf-8b3f-73cb5f4c0633 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.102764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdf577-1b41-4461-8be7-53ba9b0ad40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:26.104001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf577-1b41-4461-8be7-53ba9b0ad40e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.104347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf577-1b41-4461-8be7-53ba9b0ad40e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.104532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e921-a44b-40bf-8b3f-73cb5f4c0633 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:26.104912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf577-1b41-4461-8be7-53ba9b0ad40e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.105093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e921-a44b-40bf-8b3f-73cb5f4c0633 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:26.107651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e921-a44b-40bf-8b3f-73cb5f4c0633 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:26.124729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdf577-1b41-4461-8be7-53ba9b0ad40e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3dbdf577-1b41-4461-8be7-53ba9b0ad40e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:26.134770 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:26.137607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:26.139200 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:26.142123 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 281/565] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:26.143525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0ceb-fb4a-467b-afe5-41f091d54ff5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.143525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0ceb-fb4a-467b-afe5-41f091d54ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.143525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d0ceb-fb4a-467b-afe5-41f091d54ff5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.145221 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 282/566] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:26.146083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d0ceb-fb4a-467b-afe5-41f091d54ff5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:26.158277 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 285/567] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:45:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:26.160261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1357f-0a0b-4c48-8e24-03bed1efd415 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.160591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1357f-0a0b-4c48-8e24-03bed1efd415 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.160848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1357f-0a0b-4c48-8e24-03bed1efd415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.186159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b6f1357f-0a0b-4c48-8e24-03bed1efd415 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:26.188260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b6f1357f-0a0b-4c48-8e24-03bed1efd415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:26.291915 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 286/568] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:26.293472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1357f-0a0b-4c48-8e24-03bed1efd415 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:26.369946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d0ceb-fb4a-467b-afe5-41f091d54ff5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:26.390773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:26.426806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:26.427701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.428149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.429361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.429567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:26.429709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:26.431634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:26.450834 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 283/569] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:26.452434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:26.473521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1357f-0a0b-4c48-8e24-03bed1efd415 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:26.495754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:26.539803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:26.541337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.541852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.542339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.543317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:26.543812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:26.546334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:26.567679 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 287/570] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:26.567679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:26.631769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46dad2-870e-46b9-8097-d0fd57165825 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:26.702194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:26.703285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.703762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.704221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.723464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:26.725334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:26.755075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:26.756757 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:26.759273 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:26.759273 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:26.759273 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:26.759273 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 284/571] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:26.759273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:26.773749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0dd7d-fafb-4613-8467-5c0e0ad25e94 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:26.797178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:26.823378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:26.824415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:26.824863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:26.825304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:26.845870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:26.855791 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:26.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 288/572] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:26.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:26.894183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:26.939557 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9abbd-42ad-414e-8e08-512f3993e843 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:27.005429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:27.005823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:27.006354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:27.006777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:27.008093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=58594193ba6c41329a0bd1ca8205c69a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:27.008749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:27.011976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:27.018065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f8230-095c-4bea-857f-2646d7205885 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:27.023259 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 285/573] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:26 2022] GET /identity/v3/role_assignments?user.id=58594193ba6c41329a0bd1ca8205c69a&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:27.025063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:27.080197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:27.081190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:27.081585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:27.082027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:27.099935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:27.102068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:27.131595 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:27.133772 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:27.135917 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:27.135917 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:27.135917 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:27.135917 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 289/574] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:27.135917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:27.202201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:27.264697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e765bdc-dda5-4a12-90e7-448a16101d5f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:27.332903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:27.333975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:27.334456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:27.334932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:27.344318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:27.364240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:27.371673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458150f0-3a31-45a2-b8c3-465788cd7297 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:27.376459 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:27.377495 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:27.378708 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 286/575] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:27.381807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:27.461305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:27.462903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:27.463417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:27.463924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:27.466036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:27.466615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:27.469851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:27.493512 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 290/576] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:27.495231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:27.583235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa1c4d-4c77-44c0-9c6b-5ff74e9fcac3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:27.631994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:27.633047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:27.633523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:27.633987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:27.635689 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:27.636222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:27.639084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:27.656418 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 287/577] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:27.658147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:27.664448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:27.687967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63437768-ea91-4801-8f40-f72b745e818d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:27.737135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:27.738239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:27.738830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:27.739321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:27.740186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:27.741372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:27.743255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:27.769088 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 291/578] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:27.769088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:27.780282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:27.861187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364f69-8c37-49a8-a7c0-fb709a4e0372 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:27.919640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:27.920802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:27.921257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:27.921695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:27.922655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:27.923128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:27.925567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:27.944664 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 288/579] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:27.944664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:27.955150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250b7c8-047f-43b6-aa8f-8855b56eaf7b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:28.032327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:28.033823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:28.034473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:28.035431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:28.049001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:28.051684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:28.074754 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:28.076567 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:28.078245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:28.081194 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 292/580] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:28.083176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:28.101851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:28.137689 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f20105-8a09-423f-8080-5bc496c64c32 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:28.175133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:28.201875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:28.201875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:28.201875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:28.201875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:28.222134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:28.224983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:28.256228 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:28.258353 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:28.258353 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:28.258353 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:28.258353 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 289/581] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:28.258353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:28.283336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2daa129-477a-4142-9afd-efb96a4e80bb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:28.358770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:28.359847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:28.360279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:28.360802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:28.376433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:28.379107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:28.402444 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:28.404195 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 293/582] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:28.406575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:28.443180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc514ad2-e8e9-4161-8399-0f298f5ece88 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:28.513213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:28.514410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:28.515170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:28.515740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:28.539214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:28.542855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:28.543995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:28.572994 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:28.575387 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:28.578027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:28.578027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:28.578027 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:28.578027 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 290/583] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:28.578027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:28.612565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887bcc8-98c8-4c6a-9839-fd754ef7223b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:28.616241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:28.685059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:28.686324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:28.687002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:28.687619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:28.688953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:28.689430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:28.692270 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:28.730270 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 294/584] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:28.730270 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:28.757029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e22c3-56ad-4bbc-bf4e-5e64ff7e18ea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:28.826248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:28.827115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:28.827534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:28.827941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:28.828843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:28.829433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:28.831192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:28.855719 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 291/585] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:28.855719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:28.953838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6d1eb-6de0-4082-97f9-9185594f3af2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:29.010791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:29.029244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:29.030066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:29.030554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:29.031059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:29.032522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:29.033511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:29.034820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:29.035834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:29.069051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309a532-a691-44ee-8dd5-f6380ebf130c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:29.074108 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 295/586] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Sep 25 00:45:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:29.075406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:29.139293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:29.140097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:29.140513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:29.140858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:29.141742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:29.142167 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:29.144283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:29.164606 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 292/587] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:29.164606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:29.284441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430db5-5054-4805-995e-c7ea3ab546e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:29.366457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:29.367694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:29.368202 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:29.368811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/9d8bf513dc544632a09a2d10988a888f/roles/33e3a06092a146018396b00ae577d199` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:29.393137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d133-1893-4a97-86b6-9bccd4c383e7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:29.400200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=9d8bf513dc544632a09a2d10988a888f, role_id=33e3a06092a146018396b00ae577d199)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:29.403163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:29.428369 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 296/588] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:45:29 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/9d8bf513dc544632a09a2d10988a888f/roles/33e3a06092a146018396b00ae577d199 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:45:29.429960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:29.481869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:29.483055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:29.483645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:29.484151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:29.500922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:29.503886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:29.529121 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:29.531208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 25 00:45:29.533511 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:29.536948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:29.539069 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 293/589] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:29.540345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:29.548875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:29.705601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea210b8-52cf-4268-a914-8fdb0117020e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:29.787581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:29.789599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:29.791365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:29.791823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:29.796297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38baaf-527f-4a0b-8105-ce77fcdbc49c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:29.819481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:29.822619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:29.852548 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:29.854922 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:29.857555 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:29.857555 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:29.857555 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:29.857555 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 297/590] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:29.861331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:29.875946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:29.877793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:29.878629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:29.879491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:29.897313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:29.900783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:29.927384 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:29.929576 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 25 00:45:29.931512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:29.935509 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 294/591] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:29.937786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:30.058879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:30.076962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:30.085695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e80a7f-f2d6-4512-89e7-ccedfadc21a5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:30.154306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:30.155368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.155868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.157639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:30.158070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:30.160681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:30.163865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86be1ee-e2fe-43fd-8931-8737f9b234ca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:30.201012 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 298/592] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Sep 25 00:45:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:30.202536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:30.239818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:30.241558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.242464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.243309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.245395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:30.246410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:30.249488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:30.275901 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 295/593] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:30.277950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:30.416245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb7f2-da9f-4b25-a731-d91d95d37f63 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:30.475644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:30.476748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.477640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.478121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.479311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:30.479760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:30.481429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:30.482257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd5421-68ee-4a44-ab93-ddfc784c4e6f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:30.509981 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 299/594] 10.222.0.26 () {56 vars in 1139 bytes} [Sun Sep 25 00:45:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:30.511488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:30.522714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:30.524173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:30.540050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab90608-d4be-4136-9f80-936c6789cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:30.541235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab90608-d4be-4136-9f80-936c6789cc7e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.541692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab90608-d4be-4136-9f80-936c6789cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.542025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab90608-d4be-4136-9f80-936c6789cc7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.555725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab90608-d4be-4136-9f80-936c6789cc7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:30.557930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab90608-d4be-4136-9f80-936c6789cc7e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-aab90608-d4be-4136-9f80-936c6789cc7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:30.598298 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:30.600170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:30.602716 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:30.602716 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:30.602716 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:30.602716 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 296/595] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:30.603717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893a33-2104-4a58-b2f9-f8b3f4f113d7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.604108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893a33-2104-4a58-b2f9-f8b3f4f113d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.604665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893a33-2104-4a58-b2f9-f8b3f4f113d7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.607618 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 297/596] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:30.609990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97893a33-2104-4a58-b2f9-f8b3f4f113d7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:30.755730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33eb8e-68cb-46b0-af9f-cb8654235f68 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:30.795203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97893a33-2104-4a58-b2f9-f8b3f4f113d7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:30.839626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:30.841519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.842102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.842808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.843323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69862888-a5ed-4647-b1dd-5a0198c16ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:30.844334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69862888-a5ed-4647-b1dd-5a0198c16ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.844813 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69862888-a5ed-4647-b1dd-5a0198c16ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.845273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69862888-a5ed-4647-b1dd-5a0198c16ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.857869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:30.860204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69862888-a5ed-4647-b1dd-5a0198c16ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:30.860557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:30.862301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69862888-a5ed-4647-b1dd-5a0198c16ae1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-69862888-a5ed-4647-b1dd-5a0198c16ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:30.880715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:30.882648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:30.890854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:30.890854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:30.890854 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:30.891208 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:30.898606 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:30.908938 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:30.910697 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 298/597] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:30.910697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6564b-79b2-4fd3-a139-6bc2403a59eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:30.911014 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 300/598] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:30.911158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6564b-79b2-4fd3-a139-6bc2403a59eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:30.911158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6564b-79b2-4fd3-a139-6bc2403a59eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:30.911457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:30.923637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-15d6564b-79b2-4fd3-a139-6bc2403a59eb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:30.925743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-15d6564b-79b2-4fd3-a139-6bc2403a59eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:30.979406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:31.033096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:31.073108 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 299/599] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:31.074230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6564b-79b2-4fd3-a139-6bc2403a59eb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:31.120527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2163e-1559-48f4-82c1-dc9d6302fb49 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:31.183806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:31.184950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:31.185488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:31.185984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:31.187155 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:31.187700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:31.190290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:31.220341 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 301/600] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:31.220341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:31.302150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6564b-79b2-4fd3-a139-6bc2403a59eb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:31.393321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:31.394278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:31.394795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:31.395771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:31.398104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:31.399211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:31.402477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:31.413913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc3fd3-b4a6-4272-9e2e-a3b443fc0847 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:31.432057 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 300/601] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:31.433848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:31.469153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:31.475074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:31.498402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:31.499467 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:31.499957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:31.500481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:31.501980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:31.502365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:31.505907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:31.540417 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 302/602] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:31.540417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:31.653764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e88c2-839b-434a-83f8-5cd2b135dc82 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:31.720220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc13b-4fbb-48b5-92a9-cee472e00c77 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:31.722735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:31.723847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:31.724319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:31.724818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:31.738297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:31.741250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:31.765290 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:31.766634 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:31.767653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:31.768166 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 301/603] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:31.769474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:31.788333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:31.789503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:31.790006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:31.790538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/009181b0acc24632a5614482cdebb970/roles/33e3a06092a146018396b00ae577d199` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:31.818515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=009181b0acc24632a5614482cdebb970, role_id=33e3a06092a146018396b00ae577d199)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:31.827600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:31.864423 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 303/604] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:45:31 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/009181b0acc24632a5614482cdebb970/roles/33e3a06092a146018396b00ae577d199 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:45:31.864423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:31.917803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:31.959233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:31.973419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16656019-255d-48dd-8e5a-68df3b8c1120 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:32.052409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:32.054109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:32.054913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:32.055676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:32.079395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:32.082518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:32.113921 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:32.115715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:32.117562 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:32.125862 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57477487-4aeb-471c-ab8a-167fefa69f83 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:32.129010 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 302/605] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:32.130632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:32.219099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:32.220709 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:32.221525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:32.222301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:32.245309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:32.256513 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:32.258392 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:32.260655 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:32.265864 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 304/606] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:32.267905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:32.382489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9b506-d679-4305-bd6c-4c2995aa481b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:32.459495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:32.459495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:32.459495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:32.459495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:32.460628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:32.460950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:32.463886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:32.480249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:32.499004 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 303/607] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:32.500713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:32.512306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:32.530177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4dbe-ae66-419f-a8b5-f0d6b7d325af None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:32.612143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:32.613873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:32.614735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:32.615521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:32.617402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:32.618401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:32.622098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:32.648981 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 305/608] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:32.651437 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:32.723238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca242a-d795-4ddf-9f70-0ab5b7ea08c3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:32.797966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:32.798928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:32.799357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:32.799777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:32.800909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:32.801398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:32.803819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:32.821333 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 304/609] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:32.822868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:32.883509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9b208-fd65-4da0-900e-ff8f4ab6572a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:32.937616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:32.963731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:32.965517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:32.966364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:32.967098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/bddb3327380d43ef9b24277a3b308b0e/roles/33e3a06092a146018396b00ae577d199` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:32.999954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=bddb3327380d43ef9b24277a3b308b0e, role_id=33e3a06092a146018396b00ae577d199)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:33.004021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:33.021048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959511fd-6a7b-4f96-8765-ee79e208d483 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:33.035336 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 306/610] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:45:32 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/bddb3327380d43ef9b24277a3b308b0e/roles/33e3a06092a146018396b00ae577d199 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:45:33.037979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:33.049138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:33.164150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:33.166057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:33.170666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:33.171138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/8427b2d4e8c040d0a1b00e25e3b58e30/roles/33e3a06092a146018396b00ae577d199` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:33.215512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=8427b2d4e8c040d0a1b00e25e3b58e30, role_id=33e3a06092a146018396b00ae577d199)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:33.219195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:33.246410 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 305/611] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:45:32 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/8427b2d4e8c040d0a1b00e25e3b58e30/roles/33e3a06092a146018396b00ae577d199 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:45:33.357992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc82b4e-a9a0-4f11-931d-35f9d13f77e6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:33.439914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81c94c-1ed1-4be0-8bb1-2b7aa21f2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:33.441194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81c94c-1ed1-4be0-8bb1-2b7aa21f2e75 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:33.441758 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81c94c-1ed1-4be0-8bb1-2b7aa21f2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:33.442468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81c94c-1ed1-4be0-8bb1-2b7aa21f2e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:33.469061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81c94c-1ed1-4be0-8bb1-2b7aa21f2e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:33.472583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81c94c-1ed1-4be0-8bb1-2b7aa21f2e75 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ab81c94c-1ed1-4be0-8bb1-2b7aa21f2e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:33.510056 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:33.511994 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:33.513977 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:33.513977 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:33.513977 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:33.513977 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 307/612] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 476 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:33.526066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:33.530133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c00e5a-f190-4fda-adee-ee8de6d02e9d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:33.530596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c00e5a-f190-4fda-adee-ee8de6d02e9d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:33.532676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c00e5a-f190-4fda-adee-ee8de6d02e9d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:33.536825 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 308/613] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:33.545421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9349aa-0c92-4b02-8ec6-5bb42745691f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:33.545875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9349aa-0c92-4b02-8ec6-5bb42745691f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:33.546342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9349aa-0c92-4b02-8ec6-5bb42745691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:33.596738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ba9349aa-0c92-4b02-8ec6-5bb42745691f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:33.600564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba9349aa-0c92-4b02-8ec6-5bb42745691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:33.746010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:33.788292 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 309/614] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:45:33.791232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c4114-a3a2-42c8-a207-adc2b174c643 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:33.832515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:33.935735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:33.936684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:33.937161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:33.937729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:33.938961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:33.939448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:33.941971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:33.974052 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 306/615] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:33.982172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9349aa-0c92-4b02-8ec6-5bb42745691f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:34.210743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9349aa-0c92-4b02-8ec6-5bb42745691f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:34.282590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:34.283687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:34.284188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:34.284738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:34.307096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:34.309991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:34.330129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:34.333065 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:34.334426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:34.335992 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 310/616] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:45:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:34.344735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:34.517009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:34.684654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453e2be-fe8b-44f8-b89b-a42dcccbb804 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:34.699736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e3b92-d5ca-4188-85ad-4b7f814da274 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:34.786515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b6032-096b-4289-a515-df6376859b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:34.787054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b6032-096b-4289-a515-df6376859b54 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:34.787333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b6032-096b-4289-a515-df6376859b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:34.787605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b6032-096b-4289-a515-df6376859b54 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:34.788398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b6032-096b-4289-a515-df6376859b54 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:34.788743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b6032-096b-4289-a515-df6376859b54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:34.790256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b6032-096b-4289-a515-df6376859b54 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:34.804482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:34.805593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:34.806581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:34.807075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:34.839321 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 311/617] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Sep 25 00:45:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:34.841437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:34.848906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b6032-096b-4289-a515-df6376859b54 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:34.858958 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:34.860852 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:34.862586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:34.869877 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 307/618] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 541 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:34.886363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:35.022356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:35.137502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b6032-096b-4289-a515-df6376859b54 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:35.151366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:35.200436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13ad4e-e102-40c9-b30c-91fc8f018d65 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:35.244430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:35.245950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:35.246578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:35.247256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:35.269176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:35.272116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:35.302602 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:35.304788 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:35.306881 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:35.306881 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:35.306881 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:35.310032 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 312/619] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:35.317705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:35.318773 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:35.319268 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:35.319722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:35.319896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:35.320737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:35.321039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:35.323856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:35.352363 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 308/620] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:35.365785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:35.640650 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f84d2e-cc9d-40f1-8163-4c206bd62074 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:35.655328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:35.680137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b4470-090d-4a77-bf54-86f0b503c3d9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:35.711284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:35.713383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:35.714159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:35.714910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:35.716450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:35.717233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:35.720240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:35.745531 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 313/621] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:35.754976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:35.792656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:35.806705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:35.807422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:35.807707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:35.808026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:35.847140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:35.849407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:35.895826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:35.897740 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:35.899406 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:35.899406 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:35.899406 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:35.905520 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 309/622] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 544 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:35.917555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:35.965893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2494a89-e4f4-497e-b12c-83cbb5139fd0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:36.046121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e0058-5874-4a46-9e36-e5f4f7e69bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:36.047755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e0058-5874-4a46-9e36-e5f4f7e69bfc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:36.048611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e0058-5874-4a46-9e36-e5f4f7e69bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:36.049787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e0058-5874-4a46-9e36-e5f4f7e69bfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:36.064194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e0058-5874-4a46-9e36-e5f4f7e69bfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:36.067680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e0058-5874-4a46-9e36-e5f4f7e69bfc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-500e0058-5874-4a46-9e36-e5f4f7e69bfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:36.092821 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:36.094597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:36.096437 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:36.108802 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 314/623] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:36.111703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2757752-3c86-4dbc-acdf-d9c37abb6029 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:36.112657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2757752-3c86-4dbc-acdf-d9c37abb6029 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:36.113431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2757752-3c86-4dbc-acdf-d9c37abb6029 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:36.116572 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 315/624] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:36.122697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2757752-3c86-4dbc-acdf-d9c37abb6029 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:36.181218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3662bd-bb23-4983-94f3-2773593d9a07 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:36.258735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a78b03-013a-445d-9d9c-3490896d7804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:36.259971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a78b03-013a-445d-9d9c-3490896d7804 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:36.260768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a78b03-013a-445d-9d9c-3490896d7804 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:36.261380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a78b03-013a-445d-9d9c-3490896d7804 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:36.262666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a78b03-013a-445d-9d9c-3490896d7804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:36.264117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a78b03-013a-445d-9d9c-3490896d7804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:36.266573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a78b03-013a-445d-9d9c-3490896d7804 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:36.279367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:36.298495 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 310/625] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:36.302302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6439-7db5-4804-ba71-10faa15b82b5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:36.303081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6439-7db5-4804-ba71-10faa15b82b5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:36.303769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f6439-7db5-4804-ba71-10faa15b82b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:36.332355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:36.339086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2757752-3c86-4dbc-acdf-d9c37abb6029 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:36.355060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-122f6439-7db5-4804-ba71-10faa15b82b5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:36.357622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-122f6439-7db5-4804-ba71-10faa15b82b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:36.390548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:36.391372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:36.391783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:36.392149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:36.393007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:36.393396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:36.395093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:36.410742 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 316/626] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:36.411800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:36.492627 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 311/627] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:36.495053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f6439-7db5-4804-ba71-10faa15b82b5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:36.713598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e508b-f0e8-45fe-ae80-c7002a35ec54 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:36.782957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f6439-7db5-4804-ba71-10faa15b82b5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:36.852702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:36.854749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:36.855132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:36.855681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:36.856113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:36.856488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:36.856643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:36.857480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/58594193ba6c41329a0bd1ca8205c69a/roles/33e3a06092a146018396b00ae577d199` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:36.864732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:36.865514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:36.886707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:36.890253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:36.904421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=58594193ba6c41329a0bd1ca8205c69a, role_id=33e3a06092a146018396b00ae577d199)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:36.904421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:36.930896 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 312/628] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:45:36 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/58594193ba6c41329a0bd1ca8205c69a/roles/33e3a06092a146018396b00ae577d199 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:36.941767 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:36.943621 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:36.945523 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:36.946534 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 317/629] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:45:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 535 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:36.948034 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:36.961762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:37.279195 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce64c-651b-4064-a00a-fbf1ef178f20 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:37.330694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687e717-29fd-4fde-89cd-ca6c6892f5fc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:37.356835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:37.357506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:37.357802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:37.358096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:37.358910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:37.359219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:37.360734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:37.398103 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 318/630] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:45:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:37.407476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:37.409813 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:37.411401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:37.412241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:37.413088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:37.438636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:37.451807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:37.453727 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:37.455404 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 313/631] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 511 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:37.464586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:37.557047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:37.569718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:37.644139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184be50-3138-4d37-989e-3decfb3f6c1e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:37.688133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcfe45-2b69-4d80-9571-64cdec93791a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:37.716075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b4bea-d964-47b7-be18-905ea076456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:37.717147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b4bea-d964-47b7-be18-905ea076456f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:37.717611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b4bea-d964-47b7-be18-905ea076456f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:37.718085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b4bea-d964-47b7-be18-905ea076456f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:37.757390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3299cb-deec-4334-b43b-240e2729b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:37.759029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3299cb-deec-4334-b43b-240e2729b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:37.759029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3299cb-deec-4334-b43b-240e2729b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:37.759415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3299cb-deec-4334-b43b-240e2729b4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:37.760703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3299cb-deec-4334-b43b-240e2729b4c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:37.761246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3299cb-deec-4334-b43b-240e2729b4c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:37.764143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3299cb-deec-4334-b43b-240e2729b4c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:37.788455 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 314/632] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:37.788455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816dcfb-8e7c-4292-aa82-e94820c119e0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:37.788455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816dcfb-8e7c-4292-aa82-e94820c119e0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:37.788455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816dcfb-8e7c-4292-aa82-e94820c119e0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:37.788455 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 315/633] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:37.789189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281505-1960-46f4-a013-7fe0711c48e9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:37.789344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281505-1960-46f4-a013-7fe0711c48e9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:37.790026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281505-1960-46f4-a013-7fe0711c48e9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:37.792260 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 316/634] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:37.796280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281505-1960-46f4-a013-7fe0711c48e9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:37.860936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b4bea-d964-47b7-be18-905ea076456f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:37.863514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b4bea-d964-47b7-be18-905ea076456f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:37.952711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:38.006307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281505-1960-46f4-a013-7fe0711c48e9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:38.022503 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 319/635] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 25 00:45:38.022503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ca9f7-22bb-4179-9b55-4b3adf74a1d2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:38.022503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ca9f7-22bb-4179-9b55-4b3adf74a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:38.022503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ca9f7-22bb-4179-9b55-4b3adf74a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:38.023497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:38.044258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-527ca9f7-22bb-4179-9b55-4b3adf74a1d2 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:38.045740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-527ca9f7-22bb-4179-9b55-4b3adf74a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:38.074175 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaae06-63ac-486d-aa48-601654de239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:38.075251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaae06-63ac-486d-aa48-601654de239e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:38.075744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaae06-63ac-486d-aa48-601654de239e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:38.076224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaae06-63ac-486d-aa48-601654de239e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:38.096519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaae06-63ac-486d-aa48-601654de239e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:38.099438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaae06-63ac-486d-aa48-601654de239e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9acaae06-63ac-486d-aa48-601654de239e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:38.128600 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:38.129653 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:38.130610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:38.130610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:38.130610 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:38.136942 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 317/636] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:38.137813 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 320/637] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:38.139285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80bdad-6b2e-4acc-9e91-d1ee80e27994 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:38.139785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80bdad-6b2e-4acc-9e91-d1ee80e27994 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:38.140277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80bdad-6b2e-4acc-9e91-d1ee80e27994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:38.146001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ca9f7-22bb-4179-9b55-4b3adf74a1d2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:38.184023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fc80bdad-6b2e-4acc-9e91-d1ee80e27994 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:38.187151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fc80bdad-6b2e-4acc-9e91-d1ee80e27994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:38.321149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:38.356979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ca9f7-22bb-4179-9b55-4b3adf74a1d2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:38.367665 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 318/638] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:38.370537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80bdad-6b2e-4acc-9e91-d1ee80e27994 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:38.418670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:38.419587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:38.420061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:38.420593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:38.440143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:38.442844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:38.472504 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:38.474768 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:38.477807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:38.477807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:38.477807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:38.477807 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 321/639] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:38.484337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:38.574617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:38.602768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80bdad-6b2e-4acc-9e91-d1ee80e27994 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:38.675123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:38.676198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd2d15-3830-46fd-a4ed-9a24123eb8ae None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:38.676859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:38.677600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:38.678334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:38.680061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:38.681610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:38.684834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:38.710532 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 319/640] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:38.713146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:38.750294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:38.751370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:38.751848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:38.752314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:38.753493 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:38.754016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:38.756762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:38.761701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:38.789018 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 322/641] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:38.790888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:38.941653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9156-1a0c-41aa-975d-2a1bda86b5d9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:38.984108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b4140-d89b-4748-acf2-8fa50b59114d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:39.011572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.012596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.013048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.013749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.031427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:39.041185 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:39.043115 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 320/642] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:39.044904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:39.053420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.054277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.054643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.055031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.065010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:39.067198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:39.074492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:39.085792 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:39.088028 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 323/643] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:39.091518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:39.209471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:39.230682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a129c8-de94-463d-be00-3609af4ff79c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:39.301907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.302869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbde74-febc-4c8c-a016-44a36441af0a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:39.303636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.304354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.305075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.320730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:39.323487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:39.346379 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:39.348475 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 321/644] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:39.350695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:39.382564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.383669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.384204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.384796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.406469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:39.415898 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:39.417824 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 324/645] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:39.421075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:39.494215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:39.568290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a4f1f-478f-43a9-840c-adfca508b8d7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:39.569018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce7f68-3ed1-4ddb-bae2-7355796d4821 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:39.595132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:39.622340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.624106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.624371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.624908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.626147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:39.626659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:39.629239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:39.630789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.631750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.632351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.632836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.634044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:39.635059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:39.637543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:39.660462 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 322/646] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:39.660462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:39.672848 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 325/647] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:39.674147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:39.809852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b76b0-4a52-4560-b59f-06f23172bb89 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:39.849017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:39.859744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.860868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.861251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.861619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.862520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:39.862922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:39.864853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:39.867209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ddcfe-742d-468f-90ca-f05a09aad10c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:39.884909 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 323/648] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:39.884909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:39.938426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:39.939443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:39.939919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:39.940414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:39.941604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:39.942141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:39.944657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:39.960982 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 326/649] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:39.962535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:40.026945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:40.068517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc847a1-7758-4701-9857-57b191fdf12e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:40.120189 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b391c-a366-4a4b-bf89-8b40c5299072 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:40.139409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:40.140511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:40.140989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:40.141446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:40.161531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:40.164086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:40.179481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:40.180705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:40.181732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:40.182045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:40.194114 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:40.195407 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:40.197239 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:40.197239 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:40.197239 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:40.197239 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 324/650] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:40.198277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:40.289224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:40.324026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:40.326737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:40.397445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfbcf7-fcb9-462c-b722-e4851c511453 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:40.402218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:40.459899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:40.460975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:40.461445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:40.461859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:40.463295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=9d8bf513dc544632a09a2d10988a888f) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:40.463807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:40.466660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:40.478462 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 325/651] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:40 2022] GET /identity/v3/role_assignments?user.id=9d8bf513dc544632a09a2d10988a888f&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:40.479895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:40.541502 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 327/652] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:40.543409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:40.730511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6fd93-1c8b-4d09-a850-14ce364f1726 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:40.747349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:40.769993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754c1ae-a254-4592-99c4-f7f331a564e6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:40.801965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:40.802653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:40.802943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:40.803335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:40.818496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:40.820576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:40.846497 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:40.848575 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:40.850350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:40.850350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:40.850350 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:40.850350 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 326/653] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:40.851112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:40.853023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:40.854447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:40.855174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:40.855922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:40.857485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:40.858283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:40.861520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:40.866696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:40.890790 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 328/654] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:40.893365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:41.066844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a9015-45d6-4841-b054-b5659e096cae None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:41.067205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be85b5c-9b96-4922-92f1-7e846ad4a8a7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:41.124179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3256fb7-ecd1-4fcc-8db9-0797e7582760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:41.125269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256fb7-ecd1-4fcc-8db9-0797e7582760 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:41.125689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256fb7-ecd1-4fcc-8db9-0797e7582760 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:41.126104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3256fb7-ecd1-4fcc-8db9-0797e7582760 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:41.126464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45459547-2189-4d61-9c77-66e7aec6949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:41.127114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3256fb7-ecd1-4fcc-8db9-0797e7582760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:41.127290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45459547-2189-4d61-9c77-66e7aec6949c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:41.127432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45459547-2189-4d61-9c77-66e7aec6949c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:41.127562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3256fb7-ecd1-4fcc-8db9-0797e7582760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:41.127708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45459547-2189-4d61-9c77-66e7aec6949c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:41.129879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3256fb7-ecd1-4fcc-8db9-0797e7582760 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:41.147997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45459547-2189-4d61-9c77-66e7aec6949c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:41.150225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45459547-2189-4d61-9c77-66e7aec6949c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:41.160440 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 329/655] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:41.160440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc141257-d64d-43e9-a4b5-1448595a6f81 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:41.160440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc141257-d64d-43e9-a4b5-1448595a6f81 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:41.160440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc141257-d64d-43e9-a4b5-1448595a6f81 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:41.161152 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 330/656] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:41.162630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc141257-d64d-43e9-a4b5-1448595a6f81 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-45459547-2189-4d61-9c77-66e7aec6949c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:41.177287 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:41.179111 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:41.180935 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:41.180935 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:41.180935 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:41.186261 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 327/657] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:41.188694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21638b12-1984-4ddc-9d31-0ddb5a7c94ed None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:41.189209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21638b12-1984-4ddc-9d31-0ddb5a7c94ed None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:41.189425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21638b12-1984-4ddc-9d31-0ddb5a7c94ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:41.222887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-21638b12-1984-4ddc-9d31-0ddb5a7c94ed None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:41.225026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21638b12-1984-4ddc-9d31-0ddb5a7c94ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:41.232482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:41.282279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:41.355143 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 328/658] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:41.357044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21638b12-1984-4ddc-9d31-0ddb5a7c94ed None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:41.388907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc141257-d64d-43e9-a4b5-1448595a6f81 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:41.471680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:41.472837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:41.473324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:41.473913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:41.496154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:41.500227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:41.535933 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:41.537879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:41.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 331/659] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:41.542517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:41.594975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21638b12-1984-4ddc-9d31-0ddb5a7c94ed None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:41.675583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:41.676936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:41.677522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:41.678145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:41.697015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:41.699922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:41.705567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:41.730718 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:41.734100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 25 00:45:41.736514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:41.741949 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 329/660] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:41.744715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:41.767695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d3c22-4115-4067-ac20-65f8535dce43 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:41.803393 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:41.844766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:41.845808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:41.846303 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:41.846790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:41.848041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:41.848718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:41.851450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:41.884726 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 332/661] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:41.884726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:41.965966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa2160-ad7e-4692-8aa2-d89b261d43eb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:42.032291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:42.033354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:42.033851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:42.034323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:42.035378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:42.035921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:42.038682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:42.080956 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 330/662] 10.222.0.26 () {56 vars in 1139 bytes} [Sun Sep 25 00:45:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:42.080956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:42.115602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d74418-eaca-44d9-a76a-b1317c4801e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:42.170312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:42.170978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:42.171298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:42.172150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:42.173120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:42.173419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:42.175854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:42.182542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:42.216823 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 333/663] 10.222.0.26 () {56 vars in 1137 bytes} [Sun Sep 25 00:45:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:42.218671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:42.280574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:42.281773 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb494b-bcfe-4983-abfb-e5de6d62d496 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:42.368037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:42.369763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:42.370214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:42.370623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:42.395588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:42.404747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:42.406656 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 331/664] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:42.410058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:42.437568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f664db7-c8a0-4061-b316-a0dd9260736c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:42.508828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:42.510266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:42.510948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:42.511629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:42.523421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:42.527856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:42.549314 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:42.551160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:42.553160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:42.554035 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 334/665] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:42.555378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:42.612790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7515b-5284-4940-89ec-18716d19422a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:42.638526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:42.663860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:42.664725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:42.665134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:42.666107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:42.761653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:42.776534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89308c5e-6696-4a53-80bf-418c3f343b38 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:42.804018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:42.807269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:42.846083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:42.846737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:42.847017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:42.847443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:42.979482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:42.982242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:43.011516 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 332/666] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:43.011516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:43.038177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:43.162273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:43.172287 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 335/667] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:43.173361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:43.192537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecb4ce-5180-4974-8615-733a3898e5ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:43.263013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:43.264333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:43.264870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:43.265367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:43.266601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:43.267163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:43.269853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:43.282883 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 333/668] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:43.284269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:43.359292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0c898-bf5c-415f-939f-d3855804de1b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:43.413741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:43.414377 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:43.414659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:43.414919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:43.416191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:43.416618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:43.418134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:43.430732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:43.439698 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 336/669] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:43.441682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:43.464417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378b99c-50cd-483e-a307-14bbbc06effe None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:43.516550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:43.517254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:43.517534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:43.517914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:43.537127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:43.539357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:43.560833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:43.563726 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:43.566086 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:43.568359 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:43.568359 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:43.568359 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:43.568359 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 334/670] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:43.568359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:43.655823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b89bb-b55d-4570-8793-f37e02ed6017 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:43.702919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:43.703670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:43.704016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:43.704363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:43.718436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:43.720309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:43.738394 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:43.741122 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:43.744920 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:43.744920 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:43.744920 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:43.744920 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 337/671] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:43.744920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:43.760535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2530-db49-493a-b9d9-e4d0801f2812 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:43.803669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:43.811043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:43.812047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:43.812196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:43.812663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:43.830271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:43.832684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:43.858521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:43.860640 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:43.862601 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:43.862601 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:43.862601 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:43.862601 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 335/672] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:43.862601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:43.913573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99295d-e381-428b-a243-d113d50102bc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:43.928192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:43.982992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef40d2-c422-4938-a64b-90a38a84bedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:43.984062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef40d2-c422-4938-a64b-90a38a84bedb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:43.984610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef40d2-c422-4938-a64b-90a38a84bedb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:43.985105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef40d2-c422-4938-a64b-90a38a84bedb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:43.986537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef40d2-c422-4938-a64b-90a38a84bedb None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=009181b0acc24632a5614482cdebb970) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:43.987099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef40d2-c422-4938-a64b-90a38a84bedb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:43.990004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef40d2-c422-4938-a64b-90a38a84bedb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:44.002093 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 338/673] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:43 2022] GET /identity/v3/role_assignments?user.id=009181b0acc24632a5614482cdebb970&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:44.004343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255073-425d-42c6-b091-0b47e3fa0c9b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:44.004847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255073-425d-42c6-b091-0b47e3fa0c9b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:44.005327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b255073-425d-42c6-b091-0b47e3fa0c9b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:44.008025 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 339/674] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:44.009240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b255073-425d-42c6-b091-0b47e3fa0c9b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:44.060261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee970828-344d-4457-9e66-cea79df63c63 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:44.140827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:44.141847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:44.142399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:44.143040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:44.144622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:44.145296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:44.148202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:44.175179 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 336/675] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:44.177065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:44.236768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b255073-425d-42c6-b091-0b47e3fa0c9b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:44.287662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:44.320009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f79d6c-80ae-4592-ae72-8784cd18d459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:44.321136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79d6c-80ae-4592-ae72-8784cd18d459 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:44.321603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79d6c-80ae-4592-ae72-8784cd18d459 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:44.322628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79d6c-80ae-4592-ae72-8784cd18d459 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:44.323794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f79d6c-80ae-4592-ae72-8784cd18d459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:44.324321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f79d6c-80ae-4592-ae72-8784cd18d459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:44.327526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f79d6c-80ae-4592-ae72-8784cd18d459 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:44.350902 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 340/676] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:44.353687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c794-fd21-450b-a8e4-18a5f099c0cc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:44.354187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c794-fd21-450b-a8e4-18a5f099c0cc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:44.354665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c794-fd21-450b-a8e4-18a5f099c0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:44.399989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4cb5c794-fd21-450b-a8e4-18a5f099c0cc None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:44.402154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4cb5c794-fd21-450b-a8e4-18a5f099c0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:44.428370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423304c1-5aae-4522-8742-ad8542d66fcc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:44.451691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:44.504798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:44.506208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:44.506703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:44.507198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:44.508417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:44.508990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:44.511724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:44.542749 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 337/677] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:44.542749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:44.545833 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 341/678] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:44.547179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5c794-fd21-450b-a8e4-18a5f099c0cc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:44.751700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5c794-fd21-450b-a8e4-18a5f099c0cc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:44.761649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d608a-649d-4294-9bdc-a6cab3c5aea6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:44.768419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:44.828461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:44.830046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:44.830848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:44.831644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:44.839501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:44.840306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:44.840726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:44.840886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:44.847582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:44.850542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:44.851758 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:44.852289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:44.868302 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:44.870739 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 338/679] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:44.872753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:44.875185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:44.883998 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 342/680] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:44.886838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:44.937277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:45.089829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc65128-3709-42df-9753-d1fb658c3af9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:45.099409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1b73d-81f5-4276-8ce3-fa79497e3d0a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:45.150061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:45.151129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:45.151604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:45.152062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:45.161868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:45.162887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:45.163366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:45.163839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:45.165198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:45.166271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:45.167256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:45.168949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:45.170084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:45.196454 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 343/681] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:45.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:45.198064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:45.199120 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:45.199726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:45.201012 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 339/682] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:45:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:45.202310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:45.248067 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:45.361505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:45.390640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6742-0982-404c-a847-c026271c3a72 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:45.396134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5d80d-efe1-41a1-8648-bc7ad74d7f6a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:45.445773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:45.446859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:45.447335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:45.447807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:45.449108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:45.449614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:45.452533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:45.454550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:45.455819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:45.456318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:45.456840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:45.479877 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 344/683] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:45.479877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:45.480275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:45.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:45.490385 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:45:45.491271 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:45.497555 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 340/684] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:45:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:45.499264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:45.658033 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:45.684151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6cc91-7569-4254-8d75-78b8bbe423a2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:45.714186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30896f09-6d7d-4302-9e1f-9ed664654799 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:45.766710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:45.767705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:45.768235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:45.768736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:45.770132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=8427b2d4e8c040d0a1b00e25e3b58e30) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:45.770579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:45.773399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:45.779521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:45.780617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:45.781298 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:45.781876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:45.782982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:45.783615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:45.784332 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 345/685] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:45 2022] GET /identity/v3/role_assignments?user.id=8427b2d4e8c040d0a1b00e25e3b58e30&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:45.785795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:45.786846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:45.798263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:45.831090 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 341/686] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Sep 25 00:45:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:45.832450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:46.032791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cafc4-0a4a-4add-bb9a-ca035711990f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:46.065453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417deb-9b34-4942-a363-b582d2dfce49 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:46.089229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:46.090449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:46.090931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:46.091400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:46.092409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=bddb3327380d43ef9b24277a3b308b0e) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:46.093123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:46.094854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:46.102939 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 346/687] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:45 2022] GET /identity/v3/role_assignments?user.id=bddb3327380d43ef9b24277a3b308b0e&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:46.104237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:46.146980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:46.146980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:46.146980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:46.146980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:46.147427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:46.147427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:46.150742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:46.169576 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 342/688] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:46.179704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:46.188670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:46.231538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:46.308034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e593-5a65-4b82-8aeb-6d48518f4a78 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:46.405743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c8099-0652-4a66-b190-db1361c74dbb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:46.417621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:46.418668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:46.419167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:46.419637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:46.482205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:46.483318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:46.483866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:46.484140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:46.502394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:46.503963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:46.526622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:46.528555 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:46.531042 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:46.531042 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:46.531042 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:46.531042 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 343/689] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:46.537173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:46.590960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:46.593790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:46.656480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:46.723410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:46.756037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09d2d1-e472-4c45-97a2-7bc3054eeea6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:46.799703 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 347/690] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:46.809502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:46.841066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:46.842596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:46.843393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:46.844132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:46.845713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:46.846719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:46.849869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:46.879092 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 344/691] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:46.890811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:47.045364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fe4d6-9f1d-4135-b4b8-043d70e03100 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:47.115399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:47.116276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:47.116813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:47.117307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:47.143527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:47.143527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:47.176700 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:47.179080 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:47.181160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:47.181160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:47.181160 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:47.181160 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 348/692] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:47.189768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:47.253054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:47.356754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8f23-436f-4b8c-8e4f-e01b1ee3675a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:47.411157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:47.456330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:47.457956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:47.458443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:47.459724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:47.476913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:47.479810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:47.518331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c91105-92e2-4470-b948-78f56ff22469 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:47.522304 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:47.524312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:47.526747 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 345/693] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:45:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 635 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:47.538096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:47.618330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd0228-fdf6-404f-bb41-5afc1d1b46b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:47.619878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd0228-fdf6-404f-bb41-5afc1d1b46b1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:47.620335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd0228-fdf6-404f-bb41-5afc1d1b46b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:47.620848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd0228-fdf6-404f-bb41-5afc1d1b46b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:47.621997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd0228-fdf6-404f-bb41-5afc1d1b46b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:47.622500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd0228-fdf6-404f-bb41-5afc1d1b46b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:47.625109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd0228-fdf6-404f-bb41-5afc1d1b46b1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:47.652576 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 349/694] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:47.655025 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe3cd-2559-49aa-bb08-05bdaff3d9ac None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:47.655507 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe3cd-2559-49aa-bb08-05bdaff3d9ac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:47.655962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe3cd-2559-49aa-bb08-05bdaff3d9ac None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:47.662890 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 350/695] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:47.664371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbe3cd-2559-49aa-bb08-05bdaff3d9ac None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:47.796661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d9c7e-c26b-4832-8fdd-3546dcd00f33 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:47.835549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:47.877860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbe3cd-2559-49aa-bb08-05bdaff3d9ac None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:47.890354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa3773-20b3-4a5b-b1c1-306fc4d8c15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:47.891907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa3773-20b3-4a5b-b1c1-306fc4d8c15a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:47.893739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa3773-20b3-4a5b-b1c1-306fc4d8c15a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:47.894669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa3773-20b3-4a5b-b1c1-306fc4d8c15a None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:47.896554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa3773-20b3-4a5b-b1c1-306fc4d8c15a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:47.898369 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa3773-20b3-4a5b-b1c1-306fc4d8c15a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:47.902776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa3773-20b3-4a5b-b1c1-306fc4d8c15a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:47.949937 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 346/696] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:45:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:47.953013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333b3fa-fea0-4f77-8b30-a266b359f7ae None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:47.953916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333b3fa-fea0-4f77-8b30-a266b359f7ae None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:47.954681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333b3fa-fea0-4f77-8b30-a266b359f7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:47.962029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:47.963326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:47.963912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:47.964513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:47.979319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:47.982246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:47.992634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4333b3fa-fea0-4f77-8b30-a266b359f7ae None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:47.995674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4333b3fa-fea0-4f77-8b30-a266b359f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:48.004551 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:48.006901 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:48.009872 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:48.018171 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 351/697] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:48.020037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:48.069237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:48.151687 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 347/698] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:48.153525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333b3fa-fea0-4f77-8b30-a266b359f7ae None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:48.261625 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bb559-e990-421c-b10b-0b213fd1bf0e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:48.327989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:48.328905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:48.329294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:48.329693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:48.359950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:48.393285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333b3fa-fea0-4f77-8b30-a266b359f7ae None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:48.483370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:48.485001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:48.485781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:48.488785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:48.490545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:48.494969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:48.498165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:48.543736 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 348/699] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:48.553002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:48.597330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:48.601919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:48.667386 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:48.838553 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 352/700] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:48.841467 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:48.881465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cd317-f7c0-4a9c-82b2-b2b37a0bcb48 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:48.942198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:48.962451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:48.963685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:48.964421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:48.965078 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:48.966597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=58594193ba6c41329a0bd1ca8205c69a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:48.967246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:48.970480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:48.985212 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 349/701] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:48 2022] GET /identity/v3/role_assignments?user.id=58594193ba6c41329a0bd1ca8205c69a&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 608 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:48.987302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:49.091868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2662b2-ea26-41d8-88f9-42d3dc651c8e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:49.204423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:49.204423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:49.204423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:49.204423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:49.205110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:49.205110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:49.217426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:49.219481 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:49.221212 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 353/702] 10.222.0.26 () {56 vars in 1129 bytes} [Sun Sep 25 00:45:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:49.226394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:49.262051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:49.280695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df94a2-ba3b-4940-a9be-5bf67cd3d13d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:49.382609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:49.384188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:49.385534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:49.386357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:49.414854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:49.420936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:49.426974 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8431f-fdee-4000-8f10-b4797fb9f2af None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:49.430778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:49.459014 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:49.460907 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:49.462814 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:49.462814 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:49.462814 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:49.470817 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 350/703] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:49.474933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:49.481034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:49.481692 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:49.481977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:49.482257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:49.483431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:49.483743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:49.485305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:49.492948 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 354/704] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Sep 25 00:45:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:49.500448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:49.651696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2a315-ef50-4e59-8245-9a22ba2d1d99 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:49.722805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:49.724260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:49.724801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:49.725293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:49.726548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:49.727068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:49.730751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:49.735326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef26ec-1cb6-49e8-9e69-4c68e62e053a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:49.751543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:49.760656 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 351/705] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:49.768191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:49.823338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:49.824834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:49.825542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:49.826224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:49.828071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:49.828904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:49.832964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:49.848622 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 355/706] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Sep 25 00:45:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:49.997603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2100d6f-8fe6-4c26-ad96-620124b8ba5c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:50.047985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe921ace-ec48-4e75-8e8a-12b6bea3991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:50.049017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe921ace-ec48-4e75-8e8a-12b6bea3991d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:50.049455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe921ace-ec48-4e75-8e8a-12b6bea3991d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:50.049878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe921ace-ec48-4e75-8e8a-12b6bea3991d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:50.059037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe921ace-ec48-4e75-8e8a-12b6bea3991d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:50.061378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe921ace-ec48-4e75-8e8a-12b6bea3991d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fe921ace-ec48-4e75-8e8a-12b6bea3991d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:50.082524 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:50.083597 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:50.088075 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:50.090787 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 352/707] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:50.099392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:50.127340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766d858-dd67-4794-905a-4eae72f8c18c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:50.127868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766d858-dd67-4794-905a-4eae72f8c18c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:50.128401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766d858-dd67-4794-905a-4eae72f8c18c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:50.130073 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 353/708] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:45:50.139105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41c2f0-f977-47c1-b437-d069f221edd9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:50.139632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41c2f0-f977-47c1-b437-d069f221edd9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:50.140112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41c2f0-f977-47c1-b437-d069f221edd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:50.169651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:50.182438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9a41c2f0-f977-47c1-b437-d069f221edd9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:50.189947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9a41c2f0-f977-47c1-b437-d069f221edd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:50.287494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:50.306437 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 354/709] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:50.362322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75924306-03ea-496f-b214-3a2b42634f9b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:50.442670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e77b2-c225-4d48-bc91-c7265d9c3284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:50.443840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e77b2-c225-4d48-bc91-c7265d9c3284 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:50.444356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e77b2-c225-4d48-bc91-c7265d9c3284 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:50.444953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e77b2-c225-4d48-bc91-c7265d9c3284 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:50.446224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e77b2-c225-4d48-bc91-c7265d9c3284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:50.447391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e77b2-c225-4d48-bc91-c7265d9c3284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:50.450242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e77b2-c225-4d48-bc91-c7265d9c3284 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:50.475312 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 356/710] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:50.481569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41c2f0-f977-47c1-b437-d069f221edd9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:50.659227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41c2f0-f977-47c1-b437-d069f221edd9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:50.711925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:50.713755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:50.714262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:50.714733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:50.715682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=9d8bf513dc544632a09a2d10988a888f) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:50.716218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:50.717162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95f14-aa79-4812-bdad-493a8a5cc3d5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:50.718176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:50.718286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95f14-aa79-4812-bdad-493a8a5cc3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:50.719160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95f14-aa79-4812-bdad-493a8a5cc3d5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:50.722523 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 357/711] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:50.726019 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 355/712] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:50 2022] GET /identity/v3/role_assignments?user.id=9d8bf513dc544632a09a2d10988a888f&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:50.732425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e829afe-1f3b-4780-af8d-caeb43e5ce2d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:50.732425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e829afe-1f3b-4780-af8d-caeb43e5ce2d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:50.732425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e829afe-1f3b-4780-af8d-caeb43e5ce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:50.778337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2e829afe-1f3b-4780-af8d-caeb43e5ce2d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:50.780261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2e829afe-1f3b-4780-af8d-caeb43e5ce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:50.954983 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 358/713] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:51.016001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:51.139207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:51.250311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebe27e-8a06-49d7-8876-0722e99b8b99 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:51.318367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e829afe-1f3b-4780-af8d-caeb43e5ce2d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:51.328299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:51.329921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:51.330701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:51.331477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:51.332922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:51.335957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:51.356688 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:51.358748 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:51.360559 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 356/714] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:51 2022] GET /identity/v3/services/network => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:51.363248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:51.415735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:51.648595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba886aed-9656-4985-bffa-fc2a55fc49c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:51.688021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:51.725226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:51.726304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:51.726777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:51.727259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:51.729017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:51.729561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:51.732135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:51.741538 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 357/715] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:45:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:51.744057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e829afe-1f3b-4780-af8d-caeb43e5ce2d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:51.756164 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:51.891733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:51.892591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:51.892732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:51.893201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:51.926743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:51.937694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:51.959825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ee73c-68bd-4abf-8e72-dd047821fbaf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:51.982476 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:51.984354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:51.986159 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:51.986159 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:51.986159 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:51.992819 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 359/716] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 667 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:51.995347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:52.041553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:52.043113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:52.043935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:52.044672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:52.046016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] RBAC: Adding query filter params (type=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:52.046778 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:52.058055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:52.069660 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 358/717] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:45:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:52.168538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:52.222594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc9239-aa59-4b19-ac2d-8ff862394df1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:52.292900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bf46f-d670-4cb5-8f76-db0776783270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:52.294548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf46f-d670-4cb5-8f76-db0776783270 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:52.295038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf46f-d670-4cb5-8f76-db0776783270 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:52.295528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf46f-d670-4cb5-8f76-db0776783270 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:52.296793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bf46f-d670-4cb5-8f76-db0776783270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:52.297329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bf46f-d670-4cb5-8f76-db0776783270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:52.300170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bf46f-d670-4cb5-8f76-db0776783270 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:52.329828 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 360/718] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:52.338215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:52.467429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:52.609153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3117-c68c-4b0f-9070-f823275f945e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:52.705933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33d12e-17b5-4272-a65d-74dd7cfc2550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:52.708436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33d12e-17b5-4272-a65d-74dd7cfc2550 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:52.709321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33d12e-17b5-4272-a65d-74dd7cfc2550 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:52.710122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33d12e-17b5-4272-a65d-74dd7cfc2550 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:52.711620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33d12e-17b5-4272-a65d-74dd7cfc2550 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:52.715417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33d12e-17b5-4272-a65d-74dd7cfc2550 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:52.766938 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 359/719] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:45:52 2022] POST /identity/v3/users => generated 254 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:53.474859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602515d3-d414-4d45-9a72-b3534911b07e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:53.475652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602515d3-d414-4d45-9a72-b3534911b07e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:53.475836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602515d3-d414-4d45-9a72-b3534911b07e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:53.478273 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 361/720] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:53.488853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bbc8-d11d-4158-a83c-4762e1e8d0c0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:53.489808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bbc8-d11d-4158-a83c-4762e1e8d0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:53.490581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046bbc8-d11d-4158-a83c-4762e1e8d0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:53.528840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1046bbc8-d11d-4158-a83c-4762e1e8d0c0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:53.531329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1046bbc8-d11d-4158-a83c-4762e1e8d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:53.692914 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 360/721] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:54.273484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a1b6-3d43-409f-869e-e30cb14a5bd0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:54.274452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a1b6-3d43-409f-869e-e30cb14a5bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:54.275731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a1b6-3d43-409f-869e-e30cb14a5bd0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:54.279070 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 362/722] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:54.284150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff140c41-e84a-4bab-a0af-5f471eff78ac None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:54.284322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046bbc8-d11d-4158-a83c-4762e1e8d0c0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:54.285267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff140c41-e84a-4bab-a0af-5f471eff78ac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:54.285760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff140c41-e84a-4bab-a0af-5f471eff78ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:54.321386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ff140c41-e84a-4bab-a0af-5f471eff78ac None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:54.321386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ff140c41-e84a-4bab-a0af-5f471eff78ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:54.345823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:54.492090 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 363/723] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:54.522217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046bbc8-d11d-4158-a83c-4762e1e8d0c0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:54.590010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:54.591067 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:54.591560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:54.592039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:54.617604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:54.623787 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:54.624849 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:45:54.625758 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:54.628609 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 361/724] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:54.638152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff140c41-e84a-4bab-a0af-5f471eff78ac None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:54.813316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:54.874701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:54.923284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff140c41-e84a-4bab-a0af-5f471eff78ac None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:55.043946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:55.044875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.045301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:55.045583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:55.046330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:55.046638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:55.048295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:55.076420 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 364/725] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:55.090576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:55.145146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:55.172180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865e7ad9-cdaa-4344-9b3b-5cf004afba10 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:55.248305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3f0a0-4656-4f87-9adf-cd2095488461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:55.248980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3f0a0-4656-4f87-9adf-cd2095488461 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.249267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3f0a0-4656-4f87-9adf-cd2095488461 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:55.249551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3f0a0-4656-4f87-9adf-cd2095488461 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:55.250380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3f0a0-4656-4f87-9adf-cd2095488461 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:55.252102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3f0a0-4656-4f87-9adf-cd2095488461 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:55.281182 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 362/726] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:45:54 2022] POST /identity/v3/services => generated 251 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:55.316530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7921cf-aa28-4abb-9e01-adc083f9cd48 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.317438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7921cf-aa28-4abb-9e01-adc083f9cd48 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:55.317438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7921cf-aa28-4abb-9e01-adc083f9cd48 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:55.319759 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 363/727] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:45:55.336261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450ab8-d2bb-4c2c-846c-c8a2ea4c15c8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.336606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450ab8-d2bb-4c2c-846c-c8a2ea4c15c8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:55.336847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450ab8-d2bb-4c2c-846c-c8a2ea4c15c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:55.395276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8c450ab8-d2bb-4c2c-846c-c8a2ea4c15c8 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:55.399856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c450ab8-d2bb-4c2c-846c-c8a2ea4c15c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:55.413718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b49975-703b-4936-bfc8-dce8502a3a83 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:55.488669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:55.545561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:55.546575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.547012 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:55.547449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:55.597911 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:55.600878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:55.618724 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 364/728] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:55.625624 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:55.627543 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:55.630213 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:55.630213 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:55.630213 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:55.630213 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 365/729] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 538 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:55.635281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c450ab8-d2bb-4c2c-846c-c8a2ea4c15c8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:55.733329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:55.753977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c450ab8-d2bb-4c2c-846c-c8a2ea4c15c8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:55.794270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acf1e4-9af7-4af6-87aa-f82ee50b372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:55.794970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acf1e4-9af7-4af6-87aa-f82ee50b372b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.795251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acf1e4-9af7-4af6-87aa-f82ee50b372b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:55.795522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acf1e4-9af7-4af6-87aa-f82ee50b372b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:55.796179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acf1e4-9af7-4af6-87aa-f82ee50b372b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:55.796508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acf1e4-9af7-4af6-87aa-f82ee50b372b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:55.798011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acf1e4-9af7-4af6-87aa-f82ee50b372b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:55.812095 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 365/730] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:55.818072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:55.946380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d71a-9770-48ed-834f-bbdf4fbcf8e8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:55.985469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:55.986082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.986423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:55.986700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:55.997317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:55.998540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53477797-4da6-4579-b18f-499b6b3e54fa None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:55.998848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:55.999737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53477797-4da6-4579-b18f-499b6b3e54fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:56.000618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53477797-4da6-4579-b18f-499b6b3e54fa None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:56.003419 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 366/731] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:56.011416 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:56.013529 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:56.015650 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 366/732] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:45:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:56.018742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7396b-c214-4a00-bce4-7f226d5d8d78 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:56.018940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:56.019894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7396b-c214-4a00-bce4-7f226d5d8d78 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:56.020733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7396b-c214-4a00-bce4-7f226d5d8d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:56.063241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-abc7396b-c214-4a00-bce4-7f226d5d8d78 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:56.068916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:56.071394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-abc7396b-c214-4a00-bce4-7f226d5d8d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:56.141832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece99cb3-c2a1-4ad8-b0c4-dca96f7caf37 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:56.194391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:56.195023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:56.195342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:56.195604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:56.197609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:56.197961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:56.200847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:56.216301 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 367/733] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 25 00:45:56.217251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7396b-c214-4a00-bce4-7f226d5d8d78 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:56.225352 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 367/734] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Sep 25 00:45:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:56.231535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:56.308116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:56.344938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7396b-c214-4a00-bce4-7f226d5d8d78 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:56.401405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:56.402281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:56.402571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:56.402969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:56.418416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:56.427720 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:56.429664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:56.431425 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 368/735] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:56.436820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:56.448908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:56.487085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44df82-e7ac-4c56-8614-3a18eed383d8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:56.606226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b477-b09b-4166-9b2c-94b05eb9fe01 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:56.631395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:56.632476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:56.632991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:56.633486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:56.680795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:56.696415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:56.698236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:56.698818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:56.699329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:56.700648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:56.701179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:56.704250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:56.726365 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 369/736] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:45:56.728063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:56.780570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:56.783122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:56.951055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689f4c7-02f2-46a0-837e-976a53ef47f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:56.991317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:57.015457 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 368/737] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:45:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 25 00:45:57.016967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:57.034547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5113579-8d85-4910-ba2d-9b7e73e9a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:57.035605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5113579-8d85-4910-ba2d-9b7e73e9a2b1 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:57.036040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5113579-8d85-4910-ba2d-9b7e73e9a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:57.036541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5113579-8d85-4910-ba2d-9b7e73e9a2b1 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:57.037581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5113579-8d85-4910-ba2d-9b7e73e9a2b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:57.040020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5113579-8d85-4910-ba2d-9b7e73e9a2b1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:57.065353 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 370/738] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:45:56 2022] POST /identity/v3/services => generated 240 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:45:57.067523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4ed0d-2712-48c4-bd76-8bafe5e8aaea None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:57.067961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4ed0d-2712-48c4-bd76-8bafe5e8aaea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:57.068409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4ed0d-2712-48c4-bd76-8bafe5e8aaea None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:57.071196 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 371/739] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:57.077906 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4ed0d-2712-48c4-bd76-8bafe5e8aaea None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:57.225890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:57.246224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70858a-de6c-48b0-ab2b-3734cc4104c3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:57.340973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bc437-cb3a-4bf0-b956-a3b949f266b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:57.342516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc437-cb3a-4bf0-b956-a3b949f266b4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:57.343350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc437-cb3a-4bf0-b956-a3b949f266b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:57.344151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc437-cb3a-4bf0-b956-a3b949f266b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:57.350584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4ed0d-2712-48c4-bd76-8bafe5e8aaea None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:57.367577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc437-cb3a-4bf0-b956-a3b949f266b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:57.371589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc437-cb3a-4bf0-b956-a3b949f266b4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e66bc437-cb3a-4bf0-b956-a3b949f266b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:57.406952 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:57.408911 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:57.419071 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:57.419071 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:57.419071 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:57.419071 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 369/740] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:57.419071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89faef-2e6b-4ce5-bf11-52ab7972dc1a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:57.419071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89faef-2e6b-4ce5-bf11-52ab7972dc1a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:57.419071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89faef-2e6b-4ce5-bf11-52ab7972dc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:57.429960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:57.431829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:57.432334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:57.432878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:57.447423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2d89faef-2e6b-4ce5-bf11-52ab7972dc1a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:57.449130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2d89faef-2e6b-4ce5-bf11-52ab7972dc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:57.454522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:57.457406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:57.487259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:57.490859 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:57.495003 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:57.495003 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:57.495003 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:57.495003 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 372/741] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:57.495003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:57.506022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:57.575850 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 370/742] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:45:57.577918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89faef-2e6b-4ce5-bf11-52ab7972dc1a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:57.667822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a10e65-d50d-43bb-b6bd-9df8b6c9cf09 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:57.678345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:57.715678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89faef-2e6b-4ce5-bf11-52ab7972dc1a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:57.742960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:57.743575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:57.752704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:57.753145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:57.753886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:57.754204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:57.755832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:57.781610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:57.782721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:57.783208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:57.783700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:57.784678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:57.785198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:57.788240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:57.788915 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 373/743] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:57.795990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:57.824863 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 371/744] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:57.829722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:57.938320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:58.055027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a703e-f6a6-4a4e-90aa-2a566ad7b94b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:58.080606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169f491-e5f2-407b-bc18-be02ff631386 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:58.124828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:58.126526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:58.127062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:58.127853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:58.139081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:58.139938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:58.141093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:58.141603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:58.141810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:58.142102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:58.155563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:58.158325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:45:58.159424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:58.160522 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:45:58.161602 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:58.164358 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 374/745] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:45:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:45:58.166091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:58.187484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:58.188573 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:58.189594 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:45:58.195345 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 372/746] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:58.197536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:58.205390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:58.301923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe5a51-2c55-490a-b711-a4f97464ec10 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:58.358453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:58.359606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:58.360095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:58.360598 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:58.376526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:45:58.382811 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:58.383868 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:45:58.385319 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:58.393386 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 375/747] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:45:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:58.395126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:58.406869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:58.420640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4deff-2344-4dd1-a5b8-262a4405e7b8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:58.464223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:58.465340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:58.465864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:58.466396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:58.467383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:58.467911 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:58.469710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:58.506893 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 373/748] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:45:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:58.509034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:58.570252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:58.601290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b2689-b5fa-406b-b15c-0199cde2890a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:58.686501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:58.687660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:58.688119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:58.688559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:58.689902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:58.690424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:58.693225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:58.703643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d59e1-bace-4a80-b51b-9258aa629b74 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:58.732503 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 376/749] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:58.732503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:58.783815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:58.785864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:58.786614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:58.787440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:58.789084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:58.789948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:58.792761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:58.816867 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 374/750] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:45:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:58.818436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:58.841647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:58.912866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8e6fb-f6a8-4697-8d94-636e3db90a51 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:58.986593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:58.987617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:58.988051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:58.988523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:59.009766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:59.010065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed7f7e-3266-4364-80a6-7fde5f7f61d6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:59.012489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:59.016460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:59.043481 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:59.045413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:59.046327 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:59.046327 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:59.046327 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:59.052984 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 377/751] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:59.054777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:59.058978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311811b-7e6c-47b6-9f94-41c7c0828d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:59.059934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311811b-7e6c-47b6-9f94-41c7c0828d57 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:59.060398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311811b-7e6c-47b6-9f94-41c7c0828d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:59.060858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311811b-7e6c-47b6-9f94-41c7c0828d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:59.062111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311811b-7e6c-47b6-9f94-41c7c0828d57 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=58594193ba6c41329a0bd1ca8205c69a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:59.062641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311811b-7e6c-47b6-9f94-41c7c0828d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:59.065254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311811b-7e6c-47b6-9f94-41c7c0828d57 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:59.080494 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 375/752] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:45:58 2022] GET /identity/v3/role_assignments?user.id=58594193ba6c41329a0bd1ca8205c69a&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:59.080494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08ee61-722b-4f49-9df5-1d67734fd857 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:59.080494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08ee61-722b-4f49-9df5-1d67734fd857 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:59.080494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08ee61-722b-4f49-9df5-1d67734fd857 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:59.081441 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 376/753] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:45:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:45:59.085243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08ee61-722b-4f49-9df5-1d67734fd857 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:59.199248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:59.256006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08ee61-722b-4f49-9df5-1d67734fd857 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:59.262993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c4e56-798a-4e4b-8f1b-a795efcee6f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:59.332605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00376fae-0146-426b-8e97-f19b6b72923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:59.333901 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00376fae-0146-426b-8e97-f19b6b72923d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:59.334523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00376fae-0146-426b-8e97-f19b6b72923d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:59.335140 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00376fae-0146-426b-8e97-f19b6b72923d None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:59.336550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00376fae-0146-426b-8e97-f19b6b72923d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:45:59.337239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00376fae-0146-426b-8e97-f19b6b72923d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:59.338006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:59.339296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:59.340093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:59.340653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00376fae-0146-426b-8e97-f19b6b72923d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:59.341021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:59.365405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:59.369798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:59.373171 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 377/754] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:45:59.375702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c68c2-c76d-4f8c-ac73-ae64d100e824 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:59.376363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c68c2-c76d-4f8c-ac73-ae64d100e824 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:59.377559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c68c2-c76d-4f8c-ac73-ae64d100e824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:45:59.408650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:59.410457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:45:59.412067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:45:59.412067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:45:59.412067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:59.415127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-547c68c2-c76d-4f8c-ac73-ae64d100e824 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:45:59.423768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-547c68c2-c76d-4f8c-ac73-ae64d100e824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:45:59.424486 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 378/755] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:59.427470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:59.528861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:59.574435 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 378/756] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:45:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:45:59.576852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c68c2-c76d-4f8c-ac73-ae64d100e824 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:59.673134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61384729-7e1c-4917-8359-7f9ccf84e7b8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:45:59.788521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:45:59.827085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:45:59.828843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:45:59.830074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:45:59.830875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:45:59.858474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:45:59.861793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:45:59.884067 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:45:59.886259 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:45:59.887900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:45:59.891706 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 379/757] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:45:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:45:59.901963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:45:59.931212 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c68c2-c76d-4f8c-ac73-ae64d100e824 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:00.013379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:00.014707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.015210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.015705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.016950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:00.017510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:00.020178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:00.070965 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 379/758] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:00.081370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:00.212987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c95fd7-ef24-4b4a-a22c-e1a9afbbc64d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:00.229583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:00.291583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:00.292301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.292638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.292893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.293582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:00.293891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:00.295485 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:00.314457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa561a05-ec7e-48d4-b1ea-b72ee30d07f5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:00.316621 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 380/759] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:45:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:00.317506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:00.381631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:00.410347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:00.411590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.412070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.412522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.427445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:00.429909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:00.453685 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:00.454765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:00.455839 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 380/760] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:46:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:00.457251 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:00.474765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba5508-8cfc-44df-b63c-4887f0137b95 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:00.522055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853e249-1eed-47e2-a8ee-41d49ab0eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:00.522908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e249-1eed-47e2-a8ee-41d49ab0eba5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.523187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e249-1eed-47e2-a8ee-41d49ab0eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.523450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e249-1eed-47e2-a8ee-41d49ab0eba5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.524161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853e249-1eed-47e2-a8ee-41d49ab0eba5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:00.526219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853e249-1eed-47e2-a8ee-41d49ab0eba5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8853e249-1eed-47e2-a8ee-41d49ab0eba5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:00.536282 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:00.541761 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:00.543305 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 381/761] 10.222.0.26 () {56 vars in 1129 bytes} [Sun Sep 25 00:46:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:00.544844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051a2c0-8cfa-487d-8724-e1aacabf6560 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.544844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051a2c0-8cfa-487d-8724-e1aacabf6560 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.544844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051a2c0-8cfa-487d-8724-e1aacabf6560 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.544844 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 382/762] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:00.544844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051a2c0-8cfa-487d-8724-e1aacabf6560 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:00.586636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:00.656172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2200423-54c9-48fa-a991-a7998715f911 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:00.703228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051a2c0-8cfa-487d-8724-e1aacabf6560 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:00.721239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:00.722350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.722834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.723304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/9d8bf513dc544632a09a2d10988a888f/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.726792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=9d8bf513dc544632a09a2d10988a888f, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:00.730233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:00.752553 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 381/763] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:46:00 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/9d8bf513dc544632a09a2d10988a888f/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:46:00.753951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a78c33-acde-40e2-ba8f-122a03356b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:00.755029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a78c33-acde-40e2-ba8f-122a03356b7b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.755532 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a78c33-acde-40e2-ba8f-122a03356b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.756014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a78c33-acde-40e2-ba8f-122a03356b7b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.757574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a78c33-acde-40e2-ba8f-122a03356b7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:00.758123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a78c33-acde-40e2-ba8f-122a03356b7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:00.758889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:00.761413 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a78c33-acde-40e2-ba8f-122a03356b7b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:00.797511 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 383/764] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Sep 25 00:46:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:00.800631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f98021-9f26-427b-aa82-52d3e315734c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:00.802046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f98021-9f26-427b-aa82-52d3e315734c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:00.802832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f98021-9f26-427b-aa82-52d3e315734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:00.846692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e1f98021-9f26-427b-aa82-52d3e315734c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:00.849434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e1f98021-9f26-427b-aa82-52d3e315734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:00.886525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:01.001042 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 384/765] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:01.002844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f98021-9f26-427b-aa82-52d3e315734c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:01.023381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19f223-1042-42ca-a016-fa286fe296b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:01.088926 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:01.148119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:01.149324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:01.150369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:01.150851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:01.151988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:01.156188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:01.158938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:01.171500 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 382/766] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Sep 25 00:46:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:01.178946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:01.422483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f98021-9f26-427b-aa82-52d3e315734c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:01.484940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a67863-716c-43cb-af9f-945c6f46a907 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:01.508990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:01.523398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:01.524477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:01.525053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:01.525545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:01.573681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:01.574782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:01.575380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:01.575924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:01.576910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:01.577448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:01.579076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:01.585589 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 383/767] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Sep 25 00:46:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:01.586909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:01.681319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:01.684840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:01.725712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:01.807192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabd1d5-0e3c-4c79-a569-f03c52412067 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:01.884129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:01.885123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:01.885614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:01.886100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] PATH_INFO: `/v3/services/network` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:01.887198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:01.890431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:01.909410 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:01.911414 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:46:01.913168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:01.916904 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 384/768] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:01 2022] GET /identity/v3/services/network => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:01.918619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:01.928461 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 385/769] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 927 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:46:01.930738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:02.000315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:02.145917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c2f8-9a3f-4808-ae72-37c78cb2241c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:02.158736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11d991-e251-4369-acae-d1ca6a7f02c0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:02.216258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:02.225070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:02.226127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:02.226549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:02.227563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:02.228761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] RBAC: Adding query filter params (interface=public, service_id=d46ef55f9eb144999e2ab4a4a87082eb, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:02.229265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:02.231643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:02.252325 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 385/770] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:46:01 2022] GET /identity/v3/endpoints?service_id=d46ef55f9eb144999e2ab4a4a87082eb&interface=public®ion_id=RegionOne => generated 194 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:02.253972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:02.255775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:02.257202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:02.257927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:02.258651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:02.260579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:02.261355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:02.264542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:02.274691 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 386/771] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:02.276589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:02.396023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0213c5-56bf-4acd-ac6a-952e9db691bb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:02.404994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:02.437566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:02.438256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:02.438531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:02.438812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:02.451812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:02.453473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:02.469651 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:02.471484 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:02.473918 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:02.473918 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:02.473918 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:02.473918 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 386/772] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:02.473918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:02.502922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1b337-714c-48c2-b148-dec195934c0d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:02.577056 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:02.579374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:02.580162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:02.581093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:02.582538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:02.585759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:02.595980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337bc4-5c3e-466a-ac23-ade98c1e7213 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:02.598636 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 387/773] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:02 2022] GET /identity/v3/services => generated 772 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:02.600580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:02.659273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:02.659924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:02.660209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:02.664444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:02.664444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] RBAC: Adding query filter params (type=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:02.664444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:02.664444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:02.669442 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 387/774] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:02.680418 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:02.728723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:02.735253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:02.838944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfb8d3-5e1d-4401-83b7-801f3193635a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:02.926354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff8819-aca3-4dae-b233-fc8037ae994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:02.927277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b158f-f031-4153-a7d2-8be0cf1c9bbb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:02.927414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8819-aca3-4dae-b233-fc8037ae994a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:02.927767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8819-aca3-4dae-b233-fc8037ae994a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:02.932426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8819-aca3-4dae-b233-fc8037ae994a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:02.932426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff8819-aca3-4dae-b233-fc8037ae994a None admin] RBAC: Adding query filter params (interface=public, service_id=7f378760d2e64caa87c3319a7c5f23fe, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:02.932426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff8819-aca3-4dae-b233-fc8037ae994a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:02.932902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff8819-aca3-4dae-b233-fc8037ae994a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:02.942681 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 388/775] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:46:02 2022] GET /identity/v3/endpoints?service_id=7f378760d2e64caa87c3319a7c5f23fe&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:02.944946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0544f3e-a5d5-40b4-b305-e18ac80ea8cb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:02.945395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0544f3e-a5d5-40b4-b305-e18ac80ea8cb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:02.945842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0544f3e-a5d5-40b4-b305-e18ac80ea8cb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:02.950722 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 389/776] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:02.951067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0544f3e-a5d5-40b4-b305-e18ac80ea8cb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:02.999496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d69b9-663e-406e-831b-b8ab5cd991a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:03.000432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d69b9-663e-406e-831b-b8ab5cd991a2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:03.000893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d69b9-663e-406e-831b-b8ab5cd991a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:03.001340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d69b9-663e-406e-831b-b8ab5cd991a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:03.002303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d69b9-663e-406e-831b-b8ab5cd991a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:03.002805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d69b9-663e-406e-831b-b8ab5cd991a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:03.007606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d69b9-663e-406e-831b-b8ab5cd991a2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:03.028922 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 388/777] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:03.030447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d2db8-4b30-4d6a-8aa3-84a6e849ec07 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:03.030923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d2db8-4b30-4d6a-8aa3-84a6e849ec07 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:03.031394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d2db8-4b30-4d6a-8aa3-84a6e849ec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:03.077604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e57d2db8-4b30-4d6a-8aa3-84a6e849ec07 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:03.081073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e57d2db8-4b30-4d6a-8aa3-84a6e849ec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:03.136897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:03.222805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0544f3e-a5d5-40b4-b305-e18ac80ea8cb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:03.303054 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 389/778] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:03.303054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d2db8-4b30-4d6a-8aa3-84a6e849ec07 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:03.346986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:03.348738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:03.349618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:03.350438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:03.353973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:03.364105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:03.392001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:03.394697 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 390/779] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:02 2022] GET /identity/v3/services => generated 772 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:03.609360 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d2db8-4b30-4d6a-8aa3-84a6e849ec07 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:03.741983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:03.743051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:03.743495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:03.743999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:03.757220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:03.760534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:03.785094 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:03.786088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:03.786989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:03.787440 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 390/780] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:03.799648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:03.906211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:04.158171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1edf-0d98-4370-8c75-3ce137f6861a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:04.210169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:04.224791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a14f55-2967-47b3-bf4a-765295c004bb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:04.267405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4745ae0-6059-4d2c-b546-4c3e7a95ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:04.268558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4745ae0-6059-4d2c-b546-4c3e7a95ea2f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:04.269342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4745ae0-6059-4d2c-b546-4c3e7a95ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:04.270150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4745ae0-6059-4d2c-b546-4c3e7a95ea2f None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:04.272185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4745ae0-6059-4d2c-b546-4c3e7a95ea2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:04.273174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4745ae0-6059-4d2c-b546-4c3e7a95ea2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:04.277548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4745ae0-6059-4d2c-b546-4c3e7a95ea2f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:04.325176 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 391/781] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 526 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:04.330140 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e948330-8ce5-416f-9684-15c44498e20f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:04.330626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e948330-8ce5-416f-9684-15c44498e20f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:04.331109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e948330-8ce5-416f-9684-15c44498e20f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:04.334680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d5f0a-9d12-473b-96e4-511fa4f2176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:04.335689 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d5f0a-9d12-473b-96e4-511fa4f2176f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:04.335805 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 392/782] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:04.336317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d5f0a-9d12-473b-96e4-511fa4f2176f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:04.336832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d5f0a-9d12-473b-96e4-511fa4f2176f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:04.337189 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e948330-8ce5-416f-9684-15c44498e20f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:04.354692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d5f0a-9d12-473b-96e4-511fa4f2176f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:04.358985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-803d5f0a-9d12-473b-96e4-511fa4f2176f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:04.366057 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:04.367195 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:04.368069 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:04.372163 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 391/783] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:04.378077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a112c7-fe0e-4933-ae53-bfb0567b840b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:04.378077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a112c7-fe0e-4933-ae53-bfb0567b840b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:04.378077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a112c7-fe0e-4933-ae53-bfb0567b840b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:04.409658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-65a112c7-fe0e-4933-ae53-bfb0567b840b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:04.411676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65a112c7-fe0e-4933-ae53-bfb0567b840b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:04.514648 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 392/784] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:04.516536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a112c7-fe0e-4933-ae53-bfb0567b840b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:04.587748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e948330-8ce5-416f-9684-15c44498e20f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:04.641460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:04.670224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:04.671217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:04.671725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:04.672282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:04.673441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=616d85945f3d41839443b011bb48c412) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:04.673994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:04.675911 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:04.683438 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 393/785] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:04 2022] GET /identity/v3/role_assignments?user.id=616d85945f3d41839443b011bb48c412&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:04.690216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a112c7-fe0e-4933-ae53-bfb0567b840b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:04.736676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:04.737732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:04.738267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:04.738752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:04.739684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:04.740223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:04.741911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:04.765701 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 393/786] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:04.774336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:04.944809 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:05.015375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5d3d-0447-42bd-a97f-e189346b2d7a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:05.092322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:05.093452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:05.093989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:05.094538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:05.113692 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:05.115583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:05.121892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:05.146716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:05.147765 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:05.148808 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:05.148808 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:05.148808 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:05.151711 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 394/787] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:05.160363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:05.331264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559eb033-e9db-4f2f-b447-f795bfb399de None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:05.377556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd680c4f-9d1e-4475-9e75-b61bf594def2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:05.399485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:05.454090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:05.454826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:05.455110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:05.455465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:05.470204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:05.471240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:05.471730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:05.472187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:05.473367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:05.473895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:05.476522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:05.494793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:05.499320 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 395/788] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:05.508323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:05.516560 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:05.517753 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:05.518989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:05.524192 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 394/789] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:46:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:05.530328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:05.569005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:05.702273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55c3ff-6d21-41d3-8dd5-b1355eaa8d50 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:05.772653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:05.773645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:05.774129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:05.774600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:05.788362 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:05.791333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:05.819522 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:05.821368 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:05.823462 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 396/790] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:46:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:05.831191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:05.904193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892bf2e-fb74-4ea0-a27a-b46c0b290d61 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:06.044223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:06.045326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:06.045802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:06.046269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:06.047545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:06.048060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:06.051103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:06.068245 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 395/791] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:46:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 540 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:06.080866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:06.145898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:06.252342 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:06.285912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e2163-f006-4651-9be5-77682b2c28b4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:06.294631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20de0c-987d-443e-9011-33208045f648 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:06.330658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:06.331404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:06.331656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:06.331921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:06.356645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:06.365310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:06.393647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:06.395460 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:06.397255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:06.397255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:06.397255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:06.398385 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 396/792] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:06.402139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:06.402775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:06.403056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:06.403314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:06.404064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:06.404726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:06.406290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:06.409071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:06.441243 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 397/793] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Sep 25 00:46:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 611 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:06.449082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:06.644432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:06.652365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0f9fa-39f4-46be-81ca-94a1bcd4f240 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:06.749945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229affff-eae7-444b-bcc7-765a0607690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:06.751098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229affff-eae7-444b-bcc7-765a0607690c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:06.751584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229affff-eae7-444b-bcc7-765a0607690c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:06.752056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229affff-eae7-444b-bcc7-765a0607690c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:06.760872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24734792-184e-46b0-824c-900d4d7c501d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:06.773953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229affff-eae7-444b-bcc7-765a0607690c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:06.776847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229affff-eae7-444b-bcc7-765a0607690c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-229affff-eae7-444b-bcc7-765a0607690c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:06.807562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:06.811626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:06.811626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:06.811626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:06.811626 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 398/794] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:06.812620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a700-ecb3-46f9-b463-c1721a0c7e45 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:06.813117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a700-ecb3-46f9-b463-c1721a0c7e45 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:06.813594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a700-ecb3-46f9-b463-c1721a0c7e45 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:06.818020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:06.825912 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 399/795] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:06.827643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a700-ecb3-46f9-b463-c1721a0c7e45 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:06.848052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475a999-7b60-4fe4-8f14-93c9567f4b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:06.850484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475a999-7b60-4fe4-8f14-93c9567f4b7b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:06.851283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475a999-7b60-4fe4-8f14-93c9567f4b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:06.852678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475a999-7b60-4fe4-8f14-93c9567f4b7b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:06.854522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475a999-7b60-4fe4-8f14-93c9567f4b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:06.855063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475a999-7b60-4fe4-8f14-93c9567f4b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:06.858393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475a999-7b60-4fe4-8f14-93c9567f4b7b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:06.888658 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 397/796] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:06.891209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7d839-a934-4228-8471-918133d5de67 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:06.891672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7d839-a934-4228-8471-918133d5de67 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:06.892128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7d839-a934-4228-8471-918133d5de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:06.923984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-12d7d839-a934-4228-8471-918133d5de67 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:06.926657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-12d7d839-a934-4228-8471-918133d5de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:07.052060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a700-ecb3-46f9-b463-c1721a0c7e45 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:07.060721 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 398/797] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:07.064225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7d839-a934-4228-8471-918133d5de67 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:07.137678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:07.139512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:07.141130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:07.142388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:07.146231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:07.147398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:07.153187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:07.202631 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 400/798] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:07.213207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:07.257917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:07.418895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7d839-a934-4228-8471-918133d5de67 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:07.477001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:07.563302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149890b-187c-4ef2-8ed0-35473c28a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:07.564343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149890b-187c-4ef2-8ed0-35473c28a6ff None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:07.564868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149890b-187c-4ef2-8ed0-35473c28a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:07.565327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149890b-187c-4ef2-8ed0-35473c28a6ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:07.597362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149890b-187c-4ef2-8ed0-35473c28a6ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:07.602481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149890b-187c-4ef2-8ed0-35473c28a6ff None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:07.613413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58fd0c-31b2-48af-baac-22ec6adf2188 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7149890b-187c-4ef2-8ed0-35473c28a6ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:07.639090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:07.641675 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:46:07.643910 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:07.645927 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 399/799] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:46:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 581 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:07.652436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42f0b7-d9cf-48ca-bd10-fe6e7c88a702 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:07.652436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42f0b7-d9cf-48ca-bd10-fe6e7c88a702 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:07.652436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42f0b7-d9cf-48ca-bd10-fe6e7c88a702 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:07.652932 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 400/800] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:07.655106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42f0b7-d9cf-48ca-bd10-fe6e7c88a702 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:07.669607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdbc30-85d5-42e9-9e3c-7ba4825c144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:07.670703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdbc30-85d5-42e9-9e3c-7ba4825c144e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:07.671207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdbc30-85d5-42e9-9e3c-7ba4825c144e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:07.671700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdbc30-85d5-42e9-9e3c-7ba4825c144e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:07.685683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdbc30-85d5-42e9-9e3c-7ba4825c144e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:07.687737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdbc30-85d5-42e9-9e3c-7ba4825c144e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-65fdbc30-85d5-42e9-9e3c-7ba4825c144e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:07.702201 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:07.704446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 401/801] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 492 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0bd19-392c-466c-8fe2-c82402a3260d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:07.706605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0bd19-392c-466c-8fe2-c82402a3260d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:07.707834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0bd19-392c-466c-8fe2-c82402a3260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:07.725720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c4f0bd19-392c-466c-8fe2-c82402a3260d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:07.726928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c4f0bd19-392c-466c-8fe2-c82402a3260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:07.819106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42f0b7-d9cf-48ca-bd10-fe6e7c88a702 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:07.846501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:07.857711 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 402/802] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:07.859253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0bd19-392c-466c-8fe2-c82402a3260d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:07.873548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:07.874616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:07.875872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:07.876352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:07.877351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:07.877841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:07.880112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:07.916739 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 401/803] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:46:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:07.918265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:07.960707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:08.090809 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0bd19-392c-466c-8fe2-c82402a3260d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:08.100747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b903ce-5802-4386-86c3-a00c1d97667b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:08.163406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:08.164612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.165156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.166223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.167315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:08.169931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:08.175636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:08.176807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.177425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.177906 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.179202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:08.179751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:08.182863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:08.187168 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:08.188478 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 25 00:46:08.189538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:08.192885 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 403/804] 10.222.0.26 () {56 vars in 1129 bytes} [Sun Sep 25 00:46:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:08.193209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:08.208938 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 402/805] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:08.210483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:08.310207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:08.367613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e79726-84a8-4d53-a12b-02b22e60044b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:08.389905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:08.402870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effb9d6-12cc-496c-82d6-97de510d93aa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:08.437353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:08.438917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.439707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.440479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.443170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:08.443804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.444086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.444359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.445051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:08.445357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:08.447265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:08.452848 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 403/806] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Sep 25 00:46:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:08.453710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:08.575888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:08.578556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:08.612424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f17557-a3a8-40ba-b059-5831da1afdfb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:08.660329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:08.676045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:08.677208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.677716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.678211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/58594193ba6c41329a0bd1ca8205c69a/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.682242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=58594193ba6c41329a0bd1ca8205c69a, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:08.685011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:08.703977 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 404/807] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:46:08 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/58594193ba6c41329a0bd1ca8205c69a/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:46:08.704848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:08.725453 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 404/808] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:46:08.728339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:08.734746 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:08.885968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a506d-6ebf-43d5-8a59-0f342ea287d9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:08.914194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41ac76-c454-4bf6-b7e1-ec5d70d829ed None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:08.953428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd974165-adea-4065-8381-a9407574dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:08.954364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd974165-adea-4065-8381-a9407574dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.954776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd974165-adea-4065-8381-a9407574dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.955174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd974165-adea-4065-8381-a9407574dfd8 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.956155 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd974165-adea-4065-8381-a9407574dfd8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:08.956648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd974165-adea-4065-8381-a9407574dfd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:08.958934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd974165-adea-4065-8381-a9407574dfd8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:08.970645 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 405/809] 10.222.0.26 () {56 vars in 1124 bytes} [Sun Sep 25 00:46:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:08.970645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959174f-0218-4ed1-9f42-39c317ade77d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.970645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959174f-0218-4ed1-9f42-39c317ade77d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.970645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959174f-0218-4ed1-9f42-39c317ade77d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.970645 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 406/810] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:08.970645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959174f-0218-4ed1-9f42-39c317ade77d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:08.974019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:08.975378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:08.975856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:08.977443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] PATH_INFO: `/v3/services/network` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:08.979140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:08.981174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:08.997898 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:08.999983 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:09.001888 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 405/811] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:08 2022] GET /identity/v3/services/network => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:09.003563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:09.017190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:09.123304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:09.147394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959174f-0218-4ed1-9f42-39c317ade77d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:09.160404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d87e9-0c91-4a99-b86c-5f73ce0a2a17 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:09.215455 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408d76a-1693-40d8-8fd2-74b0c5c502ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:09.216444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d76a-1693-40d8-8fd2-74b0c5c502ce None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:09.216917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d76a-1693-40d8-8fd2-74b0c5c502ce None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:09.217360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408d76a-1693-40d8-8fd2-74b0c5c502ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:09.227052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:09.228118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:09.228646 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:09.229129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:09.230488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:09.233196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:09.236360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408d76a-1693-40d8-8fd2-74b0c5c502ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:09.239179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0408d76a-1693-40d8-8fd2-74b0c5c502ce None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0408d76a-1693-40d8-8fd2-74b0c5c502ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:09.267810 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:09.270117 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:09.272250 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:09.272250 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:09.272250 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:09.274067 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 407/812] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:09.275968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c090bb-c7f1-45ea-9c1b-da46165966e8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:09.276252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c090bb-c7f1-45ea-9c1b-da46165966e8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:09.285179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c090bb-c7f1-45ea-9c1b-da46165966e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:09.285569 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 406/813] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:09.285569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:09.312802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-84c090bb-c7f1-45ea-9c1b-da46165966e8 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:09.314693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-84c090bb-c7f1-45ea-9c1b-da46165966e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:09.456217 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 408/814] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:09.457389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c090bb-c7f1-45ea-9c1b-da46165966e8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:09.469938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:09.527617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38580ee1-f454-4f86-af59-cf44bd14cf7a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:09.594337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:09.595025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:09.595317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:09.595612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:09.596334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:09.596669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:09.599163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:09.608512 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 407/815] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:09.609324 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:09.617846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:09.654458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c090bb-c7f1-45ea-9c1b-da46165966e8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:09.735987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:09.738254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:09.739016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:09.739730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:09.741136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:09.742031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:09.745002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:09.774021 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 409/816] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:09.779779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:09.787783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d82c00-b8cd-4f57-9d64-6a948f9414ff None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:09.902436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:09.903967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:09.905255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:09.906907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:09.908587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] RBAC: Adding query filter params (type=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:09.909151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:09.910813 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:09.922062 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 408/817] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:09.923814 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:09.951212 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:10.012344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9eb0-b45d-4193-9c39-43873ba19390 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:10.053693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:10.062619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:10.063950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:10.064517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:10.065027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:10.079451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:10.082804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:10.109807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:10.111538 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 410/818] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:10.121135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:10.144957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8fadb-a07b-42d9-ab3f-327688babe1c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:10.215390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:10.217100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:10.217978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:10.218780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:10.240677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:10.252330 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:10.254176 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:46:10.255970 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:10.258902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b54758-9c85-42a9-9d93-1d991786b033 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:10.267141 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 409/819] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:46:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:10.269187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:10.287032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:10.298479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:10.298923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:10.299288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:10.299578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:10.300365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:10.301828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:10.328558 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 411/820] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:10.329530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:10.479960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ea958-e2e2-4141-848b-57208abb7a5c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:10.510486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93472c5-7d22-413f-ada8-0b40add2308c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:10.523340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:10.523912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:10.524200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:10.524501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:10.526486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:10.526486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:10.527739 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:10.538277 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 412/821] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:46:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:10.560366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:10.570248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:10.610139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:10.611782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:10.617983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:10.618947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:10.620688 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:10.621578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:10.627503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:10.680242 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 410/822] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:10.703649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:10.704340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:10.813223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e710c-d5ef-4b16-bf59-369ac7379ad8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:10.885025 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:10.886939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:10.888616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:10.889714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:10.919291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:10.922099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:10.951744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2ff25-e4d0-4700-91a5-670f625087d9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:10.953035 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:10.954835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:10.956576 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:10.956576 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:10.956576 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:10.956576 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 413/823] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:10.961219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:11.009490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:11.010208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:11.010489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:11.010792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:11.011621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=9d8bf513dc544632a09a2d10988a888f) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:11.011932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:11.013597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:11.029220 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 411/824] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:10 2022] GET /identity/v3/role_assignments?user.id=9d8bf513dc544632a09a2d10988a888f&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:11.166564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:11.183163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d447c4-ea92-4199-a4fb-6cf5815c8c75 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:11.245954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190022e-794a-48c9-aa20-add93d1a52fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:11.247432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190022e-794a-48c9-aa20-add93d1a52fd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:11.248205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190022e-794a-48c9-aa20-add93d1a52fd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:11.248938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190022e-794a-48c9-aa20-add93d1a52fd None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:11.250340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190022e-794a-48c9-aa20-add93d1a52fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:11.251203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190022e-794a-48c9-aa20-add93d1a52fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:11.253863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190022e-794a-48c9-aa20-add93d1a52fd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:11.276876 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 414/825] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:11.283942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:11.374203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:11.478445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb679d-2ce0-4f95-9e3f-fab39fb6792d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:11.538733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d1f08-e31f-490c-a04c-bba59234d472 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:11.539043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d1f08-e31f-490c-a04c-bba59234d472 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:11.539401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d1f08-e31f-490c-a04c-bba59234d472 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:11.542665 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 415/826] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:11.548068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfe447-31ee-4804-b3e3-85f08a7d9a28 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:11.548068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfe447-31ee-4804-b3e3-85f08a7d9a28 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:11.548739 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfe447-31ee-4804-b3e3-85f08a7d9a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:11.557378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:11.558815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:11.559523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:11.560231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:11.569553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e5cfe447-31ee-4804-b3e3-85f08a7d9a28 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:11.570867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e5cfe447-31ee-4804-b3e3-85f08a7d9a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:11.577221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:11.580461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:11.602158 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:11.604113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:11.605986 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 412/827] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:46:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:11.613483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:11.686712 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 416/828] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:11.769249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd439c14-81ff-4af4-b4f3-88712c76f4d3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:11.777903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:11.808220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:11.808909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:11.809183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:11.809448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:11.810236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:11.810532 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:11.812091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:11.834351 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 413/829] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Sep 25 00:46:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:11.841744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfe447-31ee-4804-b3e3-85f08a7d9a28 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:12.003062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:12.036207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfe447-31ee-4804-b3e3-85f08a7d9a28 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:12.099506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:12.100587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:12.101068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:12.101551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:12.122524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:12.125223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:12.154440 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:12.156307 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:12.157588 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:12.157588 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:12.157588 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:12.157588 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 417/830] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:12.166490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:12.170201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:12.385870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a7eac-1af2-4cab-a647-e9343c08de13 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:12.389202 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc348dfa-a88d-4279-b06d-184f4a33be22 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:12.464915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:12.465953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:12.466395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:12.466565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:12.466877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:12.467859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:12.468411 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:12.468710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:12.469571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:12.470373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:12.470931 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:12.474666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:12.495928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:12.500460 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 414/831] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:12.506639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:12.515798 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:12.517562 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:12.519242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:12.525763 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 418/832] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:46:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:12.528763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:12.576469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:12.729098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363790dc-7a34-4d9e-b2ac-66176f27a031 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:12.737069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726a7ca-f5bb-4249-8a73-abef28f43335 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:12.797443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:12.799075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:12.799800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:12.800412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:12.807572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:12.808644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:12.809139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:12.809690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:12.810922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:12.811452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:12.814170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:12.817772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:12.819965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:12.840250 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 419/833] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:46:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:12.844121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:12.848832 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:12.849857 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:12.850872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:12.855303 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 415/834] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:12.865998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:12.972890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:13.085263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:13.088201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e1682-dc94-4f11-bb2f-dc193276db38 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:13.119882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10afa99-7324-4897-b29d-de3ca8aac03f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:13.159796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:13.160911 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:13.161356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:13.161774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:13.182546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:13.185073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:13.203749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:13.205263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:13.206061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:13.206822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:13.208419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:13.209272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:13.210973 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:13.213262 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:13.215018 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:13.215018 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:13.215018 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:13.217575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:13.219194 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 420/835] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:13.222493 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:13.254833 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 416/836] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:13.260239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:13.430915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5f50c-28f1-4f3e-b757-683680118173 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:13.445620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561bdea-b80e-4ae9-8d4d-fe5d8556c66c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:13.451987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:13.490390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:13.491236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:13.491697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:13.492118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/616d85945f3d41839443b011bb48c412/roles/33e3a06092a146018396b00ae577d199` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:13.501782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57dd29-0ef0-4ccd-aa54-bb2f46e2f6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:13.502992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57dd29-0ef0-4ccd-aa54-bb2f46e2f6cc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:13.503488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57dd29-0ef0-4ccd-aa54-bb2f46e2f6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:13.503970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57dd29-0ef0-4ccd-aa54-bb2f46e2f6cc None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:13.505291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57dd29-0ef0-4ccd-aa54-bb2f46e2f6cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:13.505905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57dd29-0ef0-4ccd-aa54-bb2f46e2f6cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:13.508699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57dd29-0ef0-4ccd-aa54-bb2f46e2f6cc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:13.512338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=616d85945f3d41839443b011bb48c412, role_id=33e3a06092a146018396b00ae577d199)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:13.514632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:13.517412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:13.544433 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 421/837] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:13.544677 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 417/838] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:46:13 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/616d85945f3d41839443b011bb48c412/roles/33e3a06092a146018396b00ae577d199 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:46:13.544770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c1a5-342d-4abb-bf84-372809d49e78 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:13.544770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c1a5-342d-4abb-bf84-372809d49e78 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:13.544770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c1a5-342d-4abb-bf84-372809d49e78 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:13.544972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:13.547296 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 422/839] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:13.549416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5601bad-3de3-4c6d-accb-ccc38e4b4f15 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:13.549696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5601bad-3de3-4c6d-accb-ccc38e4b4f15 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:13.549974 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5601bad-3de3-4c6d-accb-ccc38e4b4f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:13.583037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e5601bad-3de3-4c6d-accb-ccc38e4b4f15 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:13.585276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e5601bad-3de3-4c6d-accb-ccc38e4b4f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:13.725085 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 423/840] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:13.751171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9c7c1-6659-4377-9116-aa6dff239b4d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:13.825916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:13.827030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:13.827529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:13.828010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:13.843102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:13.845865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:13.863441 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:13.865326 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:13.867205 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 418/841] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:46:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:13.875632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5601bad-3de3-4c6d-accb-ccc38e4b4f15 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:14.064038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:14.072601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5601bad-3de3-4c6d-accb-ccc38e4b4f15 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:14.141966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:14.143092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:14.143659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:14.144216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:14.145674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:14.146280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:14.149260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:14.184948 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 424/842] 10.222.0.26 () {56 vars in 1135 bytes} [Sun Sep 25 00:46:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:14.193717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:14.234962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:14.286647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:14.366236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19992c1-634b-4572-826e-99235c1aec9f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:14.433189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:14.434356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:14.434881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:14.435368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:14.448779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b20b4e-0bc3-47c4-bfb5-73aaa3aff1f3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:14.508251 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:14.508947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:14.509315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:14.509576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:14.510169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:14.511655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:14.525629 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:14.526838 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:14.527836 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:14.529593 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 425/843] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:14.537081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:14.577826 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:14.588237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:14.593224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:14.718881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:14.725921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab953a-d10b-43b0-9251-d2420c4f090f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:14.793273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:14.794326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:14.794888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:14.795364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:14.796568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:14.797083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:14.799857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:14.815229 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 426/844] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:14.819040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:14.830631 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 419/845] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:46:14.837238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:14.968580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:14.998376 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc543a1c-cd37-4576-a361-59f99266c927 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:15.052218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509674b-2b1d-4901-a45d-4c1d82131af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:15.053359 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509674b-2b1d-4901-a45d-4c1d82131af6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:15.053855 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509674b-2b1d-4901-a45d-4c1d82131af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:15.054351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509674b-2b1d-4901-a45d-4c1d82131af6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:15.055577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509674b-2b1d-4901-a45d-4c1d82131af6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:15.056175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509674b-2b1d-4901-a45d-4c1d82131af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:15.059070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509674b-2b1d-4901-a45d-4c1d82131af6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:15.059992 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775f45-78f9-48aa-a602-f8e8d564d85f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:15.072428 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 427/846] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:15.072428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d90f7-2a03-41fd-978d-21b3beb44e00 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:15.072428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d90f7-2a03-41fd-978d-21b3beb44e00 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:15.072428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d90f7-2a03-41fd-978d-21b3beb44e00 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:15.076731 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 428/847] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:15.085121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951918a1-55fa-43ec-8f0f-11987469295c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:15.085629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951918a1-55fa-43ec-8f0f-11987469295c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:15.086274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951918a1-55fa-43ec-8f0f-11987469295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:15.122129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-951918a1-55fa-43ec-8f0f-11987469295c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:15.124123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-951918a1-55fa-43ec-8f0f-11987469295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:15.143280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:15.144296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:15.144805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:15.146241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:15.170934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:15.174436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:15.209240 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:15.211090 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:15.214006 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:15.214006 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:15.214006 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:15.214006 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 420/848] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:15.225389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:15.242917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:15.278320 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 429/849] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:15.437148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5aae9-d28f-4957-8f5a-f0e7571392ba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:15.503983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:15.505050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:15.505542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:15.506023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:15.507226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:15.507764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:15.510433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:15.534962 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 421/850] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:15.542904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951918a1-55fa-43ec-8f0f-11987469295c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:15.692788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:15.736651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951918a1-55fa-43ec-8f0f-11987469295c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:15.764958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:15.818136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:15.819122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:15.819951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:15.820459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:15.838598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:15.841363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:15.869502 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:15.871249 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:15.873047 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:15.879962 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 430/851] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:15.882214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:15.911920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:15.990659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76711548-4799-4bf1-8479-21928fb144a8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:16.069814 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:16.070795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:16.071278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:16.071845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:16.073562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:16.076105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:16.109077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0b8b4-c6ac-4cf0-9e00-6e56e335e2d9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:16.110878 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:16.112676 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:16.114390 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:16.133759 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 422/852] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Sep 25 00:46:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:16.137189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:16.184556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:16.186112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:16.186905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:16.187670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:16.189255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:16.190083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:16.193046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:16.206964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:16.228289 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 431/853] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:16.230701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:16.346943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afe73a-70e1-4b43-b423-7ecc0f135dac None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:16.411820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:16.425904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:16.427171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:16.433433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:16.433433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:16.433433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:16.433433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:16.433433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:16.451078 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 423/854] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:16.458256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:16.459449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e247e-c068-4680-9fb8-db2190fea1a7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:16.533899 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179322da-8c88-43be-bb8a-33917a4404bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:16.533899 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179322da-8c88-43be-bb8a-33917a4404bb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:16.534321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179322da-8c88-43be-bb8a-33917a4404bb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:16.534920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179322da-8c88-43be-bb8a-33917a4404bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:16.536402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179322da-8c88-43be-bb8a-33917a4404bb None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=58594193ba6c41329a0bd1ca8205c69a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:16.538565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179322da-8c88-43be-bb8a-33917a4404bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:16.542096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179322da-8c88-43be-bb8a-33917a4404bb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:16.560762 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 432/855] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:16 2022] GET /identity/v3/role_assignments?user.id=58594193ba6c41329a0bd1ca8205c69a&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:16.672165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d2c3d-fe72-4008-8494-c61d12611cd9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:16.743419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c967d-eb8b-462b-8b40-d684e70cbe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:16.745557 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c967d-eb8b-462b-8b40-d684e70cbe32 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:16.746360 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c967d-eb8b-462b-8b40-d684e70cbe32 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:16.747053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c967d-eb8b-462b-8b40-d684e70cbe32 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:16.748674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c967d-eb8b-462b-8b40-d684e70cbe32 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:16.749458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c967d-eb8b-462b-8b40-d684e70cbe32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:16.752345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c967d-eb8b-462b-8b40-d684e70cbe32 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:16.762953 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 424/856] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:16.936422 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510ca36-bebb-4fda-891e-bc9066c2cf76 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:16.936681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510ca36-bebb-4fda-891e-bc9066c2cf76 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:16.936967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510ca36-bebb-4fda-891e-bc9066c2cf76 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:16.939952 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 433/857] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:16.949329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca16da-7f75-4342-ba64-94b3867ceb08 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:16.950708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca16da-7f75-4342-ba64-94b3867ceb08 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:16.951204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca16da-7f75-4342-ba64-94b3867ceb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:16.976642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f0ca16da-7f75-4342-ba64-94b3867ceb08 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:16.978444 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f0ca16da-7f75-4342-ba64-94b3867ceb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:17.061941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:17.090070 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 425/858] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:17.373395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510ca36-bebb-4fda-891e-bc9066c2cf76 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:17.462265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:17.510208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510ca36-bebb-4fda-891e-bc9066c2cf76 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:17.552749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:17.554365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:17.555119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:17.555874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:17.579199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:17.590185 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:17.591474 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:46:17.593228 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:17.597577 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 434/859] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:17.606780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca16da-7f75-4342-ba64-94b3867ceb08 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:17.987291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca16da-7f75-4342-ba64-94b3867ceb08 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:18.055830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:18.057035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:18.057514 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:18.057986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:18.059208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:18.059737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:18.062436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:18.081082 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 426/860] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:18.092117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:18.291082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c18696-6273-4014-a39b-1c4cc5334e0f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:18.320415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:18.333614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0922-e35b-4e52-b6aa-a4c461029780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:18.334469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0922-e35b-4e52-b6aa-a4c461029780 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:18.334755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0922-e35b-4e52-b6aa-a4c461029780 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:18.335038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0922-e35b-4e52-b6aa-a4c461029780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:18.356413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0922-e35b-4e52-b6aa-a4c461029780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:18.356413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0922-e35b-4e52-b6aa-a4c461029780 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-301b0922-e35b-4e52-b6aa-a4c461029780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:18.373879 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:18.376643 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:18.378452 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:18.378452 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:18.378452 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:18.378452 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 435/861] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:18.384251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:18.489706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:18.589371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ea2c4-5de4-46bf-bef8-1181d23b76f4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:18.643750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64302b1b-0832-4ad7-900d-01ad99212490 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:18.644840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64302b1b-0832-4ad7-900d-01ad99212490 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:18.645694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64302b1b-0832-4ad7-900d-01ad99212490 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:18.649839 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 436/862] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:18.659950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da35f84-1515-4cba-b419-6244aebdc546 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:18.660925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da35f84-1515-4cba-b419-6244aebdc546 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:18.661724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da35f84-1515-4cba-b419-6244aebdc546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:18.666839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:18.667944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:18.668432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:18.668915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:18.670107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:18.670629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:18.673261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:18.699119 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 427/863] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:18.707022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:18.708650 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1da35f84-1515-4cba-b419-6244aebdc546 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:18.710852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1da35f84-1515-4cba-b419-6244aebdc546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:18.839861 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 437/864] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:18.899808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901482cc-36b0-442b-a4c0-72d38df916d1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:18.921049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:18.965876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:18.966934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:18.967409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:18.967887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:18.983419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:18.986122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:19.010106 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:19.012622 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:19.016341 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:19.018318 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 428/865] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:46:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:19.022605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da35f84-1515-4cba-b419-6244aebdc546 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:19.217824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da35f84-1515-4cba-b419-6244aebdc546 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:19.233712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:19.289170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:19.289883 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:19.290169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:19.290586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:19.291698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:19.292089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:19.294229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:19.308892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:19.324108 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 438/866] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Sep 25 00:46:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:19.333415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:19.513896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a7b5a-6c8f-4db6-9e6a-c9e7430d8d4c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:19.555009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c9bb6-1194-4fac-975c-5e9f8a2769ec None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:19.583609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1ef7-f1ff-4cd5-b43b-3f4d50c10661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:19.584372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1ef7-f1ff-4cd5-b43b-3f4d50c10661 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:19.584716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1ef7-f1ff-4cd5-b43b-3f4d50c10661 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:19.585065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1ef7-f1ff-4cd5-b43b-3f4d50c10661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:19.631076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12654493-929a-4fd8-825c-db91e78e0e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:19.632920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12654493-929a-4fd8-825c-db91e78e0e25 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:19.633661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12654493-929a-4fd8-825c-db91e78e0e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:19.634806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12654493-929a-4fd8-825c-db91e78e0e25 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:19.636468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12654493-929a-4fd8-825c-db91e78e0e25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:19.639602 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12654493-929a-4fd8-825c-db91e78e0e25 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:19.669865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:19.670379 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 429/867] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:46:19 2022] POST /identity/v3/services => generated 251 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:19.673412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf73cb-3a6f-4722-aa9e-7f1b5a2d8096 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:19.674480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf73cb-3a6f-4722-aa9e-7f1b5a2d8096 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:19.674777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf73cb-3a6f-4722-aa9e-7f1b5a2d8096 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:19.677974 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 430/868] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:19.691054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2ab0-d9c0-485c-a370-792db591c0bf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:19.691661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2ab0-d9c0-485c-a370-792db591c0bf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:19.692421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b2ab0-d9c0-485c-a370-792db591c0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:19.724578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b65b2ab0-d9c0-485c-a370-792db591c0bf None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:19.726445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b65b2ab0-d9c0-485c-a370-792db591c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:19.774775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:19.782459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c1ef7-f1ff-4cd5-b43b-3f4d50c10661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:19.786242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c1ef7-f1ff-4cd5-b43b-3f4d50c10661 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:19.889015 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 431/869] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:20.020677 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 439/870] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 25 00:46:20.028120 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b2ab0-d9c0-485c-a370-792db591c0bf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:20.106141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd35770-a398-44e2-89bd-701f40aee71d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:20.106754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd35770-a398-44e2-89bd-701f40aee71d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:20.107325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd35770-a398-44e2-89bd-701f40aee71d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:20.110258 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 440/871] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:20.118886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baced4b7-af3c-4651-9378-e80e4cf1cf82 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:20.119331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baced4b7-af3c-4651-9378-e80e4cf1cf82 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:20.119895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baced4b7-af3c-4651-9378-e80e4cf1cf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:20.152796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-baced4b7-af3c-4651-9378-e80e4cf1cf82 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:20.155759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-baced4b7-af3c-4651-9378-e80e4cf1cf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:20.233180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b2ab0-d9c0-485c-a370-792db591c0bf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:20.285009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:20.285870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:20.286218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:20.286559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:20.289174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:20.300593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:20.302385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:20.313323 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 441/872] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:20.320506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:20.322776 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:20.329413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:20.329413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:20.329413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:20.329413 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 432/873] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:20.332880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baced4b7-af3c-4651-9378-e80e4cf1cf82 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:20.442441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:20.463714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:20.521919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baced4b7-af3c-4651-9378-e80e4cf1cf82 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:20.593524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:20.594475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:20.594960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:20.595462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:20.596705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:20.597261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:20.599980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:20.624016 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 442/874] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:20.633968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:20.659847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b820f2-656f-40d1-8a12-e3b18ea4be1d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:20.733433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:20.734119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:20.734407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:20.734684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:20.735356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:20.736902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:20.753097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:20.762894 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 433/875] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:46:20 2022] POST /identity/v3/services => generated 269 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:20.807674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:20.876302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e888bc7-e776-4e0c-b721-417429c8d759 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:20.897031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:20.965333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:20.966997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:20.967797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:20.968615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:20.973614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb788e0-73af-47db-a882-73cf4ffdc26e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:20.984791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:20.988282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:21.011549 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:21.013199 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:21.015128 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:21.019113 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 443/876] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:21.029725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:21.059367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:21.061660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:21.063443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:21.064241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:21.065658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:21.068340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:21.092296 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:21.093554 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:21.094449 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:21.096456 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 434/877] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:21.106467 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:21.224247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d4ac7-5c8e-43a3-9a9a-46b3e453c298 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:21.242849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:21.290493 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:21.292090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:21.292940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:21.293706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:21.295285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:21.296015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:21.298640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:21.320939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd712c2-c759-47c3-812e-ea7bee422c98 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:21.335902 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 444/878] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:21.350663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:21.393117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:21.394528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:21.395376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:21.396208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:21.397853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:21.400730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:21.402783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:21.407470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:21.417994 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 435/879] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:21.427077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:21.570099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0929d-4e38-425b-b8d2-328b7d51b879 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:21.636257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead3055-c5c8-4b5e-8ea8-5ce3116442db None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:21.638276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de8124-3571-4b45-8faa-f7cfe43ed9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:21.639345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de8124-3571-4b45-8faa-f7cfe43ed9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:21.640055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de8124-3571-4b45-8faa-f7cfe43ed9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:21.640530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de8124-3571-4b45-8faa-f7cfe43ed9b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:21.641842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de8124-3571-4b45-8faa-f7cfe43ed9b5 None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=616d85945f3d41839443b011bb48c412) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:21.642365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de8124-3571-4b45-8faa-f7cfe43ed9b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:21.645009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de8124-3571-4b45-8faa-f7cfe43ed9b5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:21.660952 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 445/880] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:21 2022] GET /identity/v3/role_assignments?user.id=616d85945f3d41839443b011bb48c412&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:21.713133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b5f4e-628a-4474-a783-de7844764fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:21.714163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b5f4e-628a-4474-a783-de7844764fab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:21.714616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b5f4e-628a-4474-a783-de7844764fab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:21.715081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b5f4e-628a-4474-a783-de7844764fab None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:21.716169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b5f4e-628a-4474-a783-de7844764fab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:21.716882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b5f4e-628a-4474-a783-de7844764fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:21.719592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b5f4e-628a-4474-a783-de7844764fab None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:21.729990 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 436/881] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:23.140665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d42d5-141a-40a4-a289-471bfcffaa35 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:23.141205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d42d5-141a-40a4-a289-471bfcffaa35 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:23.141598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d42d5-141a-40a4-a289-471bfcffaa35 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:23.144285 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 446/882] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:23.153141 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef0c2b-5cea-4993-a4fa-a82123758f4d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:23.153641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef0c2b-5cea-4993-a4fa-a82123758f4d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:23.154042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef0c2b-5cea-4993-a4fa-a82123758f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:23.184696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-13ef0c2b-5cea-4993-a4fa-a82123758f4d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:23.186636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-13ef0c2b-5cea-4993-a4fa-a82123758f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:23.245292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:23.264046 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 437/883] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:23.747100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d42d5-141a-40a4-a289-471bfcffaa35 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:23.809710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:23.961031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d42d5-141a-40a4-a289-471bfcffaa35 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:24.028797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd9aea-1680-46af-84b0-48d03ec2dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:24.030142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd9aea-1680-46af-84b0-48d03ec2dfdf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.030790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd9aea-1680-46af-84b0-48d03ec2dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.031450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd9aea-1680-46af-84b0-48d03ec2dfdf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.032695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd9aea-1680-46af-84b0-48d03ec2dfdf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:24.035349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd9aea-1680-46af-84b0-48d03ec2dfdf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e9dd9aea-1680-46af-84b0-48d03ec2dfdf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:24.055357 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:24.057446 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:24.061745 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 447/884] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:24.069750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef0c2b-5cea-4993-a4fa-a82123758f4d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:24.112927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661db3-e58b-4aed-9394-8738f4528c42 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.113242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661db3-e58b-4aed-9394-8738f4528c42 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.113492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf661db3-e58b-4aed-9394-8738f4528c42 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.115018 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 448/885] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:24.123218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c5a6-750c-4402-8ce3-76558b1fae0f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.123613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c5a6-750c-4402-8ce3-76558b1fae0f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.123764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c5a6-750c-4402-8ce3-76558b1fae0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.144235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d940c5a6-750c-4402-8ce3-76558b1fae0f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:24.145511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d940c5a6-750c-4402-8ce3-76558b1fae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:24.236456 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 449/886] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:24.258813 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef0c2b-5cea-4993-a4fa-a82123758f4d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:24.300962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64e978-ac1d-49aa-9456-aa081c4778f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:24.301543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64e978-ac1d-49aa-9456-aa081c4778f4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.301822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64e978-ac1d-49aa-9456-aa081c4778f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.302102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64e978-ac1d-49aa-9456-aa081c4778f4 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.302750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64e978-ac1d-49aa-9456-aa081c4778f4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:24.303055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64e978-ac1d-49aa-9456-aa081c4778f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:24.304504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64e978-ac1d-49aa-9456-aa081c4778f4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:24.311046 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 438/887] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:24.317827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940c5a6-750c-4402-8ce3-76558b1fae0f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:24.350537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:24.383550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1af4b5-0893-4c30-804d-f6e256d0dc66 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.383975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1af4b5-0893-4c30-804d-f6e256d0dc66 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.384163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1af4b5-0893-4c30-804d-f6e256d0dc66 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.386022 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 439/888] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:24.393790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4c87-b84f-41d1-9418-5ad28a6eb282 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.394558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4c87-b84f-41d1-9418-5ad28a6eb282 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.395397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c4c87-b84f-41d1-9418-5ad28a6eb282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.419656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-df0c4c87-b84f-41d1-9418-5ad28a6eb282 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:24.421728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-df0c4c87-b84f-41d1-9418-5ad28a6eb282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:24.445797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940c5a6-750c-4402-8ce3-76558b1fae0f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:24.488934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:24.500057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:24.500717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.501003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.501274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.501914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:24.502213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:24.506040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:24.517059 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 450/889] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:24.523577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:24.532435 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 440/890] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:24.691603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c4c87-b84f-41d1-9418-5ad28a6eb282 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:24.710467 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:24.732471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda34be-75c6-4712-822e-ea6e86a31e88 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:24.797249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:24.797907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.798188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.798637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.799995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] RBAC: Adding query filter params (interface=public, service_id=ecf3ca7886e9498e9914fcb24f175a62, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:24.800358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:24.802212 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:24.816515 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 451/891] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:46:24 2022] GET /identity/v3/endpoints?service_id=ecf3ca7886e9498e9914fcb24f175a62&interface=public®ion_id=RegionOne => generated 194 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:24.818734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:24.864153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c4c87-b84f-41d1-9418-5ad28a6eb282 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:24.908727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:24.909451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:24.909884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:24.910185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:24.910825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:24.912595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:24.924937 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:24.926989 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:24.928822 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:24.932049 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 441/892] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Sep 25 00:46:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:24.933042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:25.004993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:25.041854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e38e8-ab47-4a9c-9c17-b09c44dcc47e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:25.087049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09642a39-d5ea-482e-a5a1-a9d583983d72 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:25.087294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:25.088265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:25.088585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:25.088864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:25.089490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:25.090889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:25.098870 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 452/893] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:24 2022] GET /identity/v3/services => generated 1268 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 00:46:25.099897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:25.140941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:25.161341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b9102-6f3b-4287-aeab-2bb8ff6297d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:25.162383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9102-6f3b-4287-aeab-2bb8ff6297d0 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:25.162872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9102-6f3b-4287-aeab-2bb8ff6297d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:25.163388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9102-6f3b-4287-aeab-2bb8ff6297d0 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:25.164658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b9102-6f3b-4287-aeab-2bb8ff6297d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:25.167555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b9102-6f3b-4287-aeab-2bb8ff6297d0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:25.191019 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 442/894] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:46:24 2022] POST /identity/v3/services => generated 241 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:25.193805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e2c1f-5324-428c-9ed5-82229d0eb3a7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:25.194656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e2c1f-5324-428c-9ed5-82229d0eb3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:25.195138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e2c1f-5324-428c-9ed5-82229d0eb3a7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:25.198826 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 443/895] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:25.207731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c86cb-06f9-40e1-a64e-9aae1ababcb1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:25.209263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c86cb-06f9-40e1-a64e-9aae1ababcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:25.209263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c86cb-06f9-40e1-a64e-9aae1ababcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:25.242442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-212c86cb-06f9-40e1-a64e-9aae1ababcb1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:25.245181 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-212c86cb-06f9-40e1-a64e-9aae1ababcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:25.323672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd6382-0faa-4613-a110-caaac03283ca None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:25.397648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:25.398760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:25.399272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:25.399773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:25.400477 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 444/896] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:25.401107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:25.401671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:25.404436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:25.414640 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 453/897] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:25.424824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c86cb-06f9-40e1-a64e-9aae1ababcb1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:25.488284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:25.612218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c86cb-06f9-40e1-a64e-9aae1ababcb1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:25.653415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:25.654120 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:25.654398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:25.654670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:25.655388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:25.655692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:25.657439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:25.664313 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 445/898] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:25.674758 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:25.762354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:25.809854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:25.833059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580e79-ae5e-4b14-a3de-9cd11d59f495 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:25.903634 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:25.904374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:25.904697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:25.905037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:25.906166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] RBAC: Adding query filter params (interface=public, service_id=52a79830a5264358b34d803c8f4a2e04, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:25.906484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:25.909477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:25.925235 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 454/899] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:46:25 2022] GET /identity/v3/endpoints?service_id=52a79830a5264358b34d803c8f4a2e04&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:25.926602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:25.949383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:25.959041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4445bb-b3e9-4ce8-bbe0-fc9ff5fbb7c9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:26.006127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:26.006785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:26.007065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:26.007335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:26.020419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:26.025715 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:26.026736 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:26.027673 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:26.032689 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 446/900] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Sep 25 00:46:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:26.034416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:26.063638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98766fb2-c422-4ae1-8c4a-dc5b2b977b97 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:26.107790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:26.108904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:26.109451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:26.109930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:26.110815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:26.112733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:26.128884 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 455/901] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:25 2022] GET /identity/v3/services => generated 1497 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 00:46:26.204344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f4d6b-5800-43b6-af23-48603c1537b2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:26.274697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:26.275811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:26.276264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:26.276787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:26.277983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:26.278518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:26.281234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:26.304695 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 447/902] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Sep 25 00:46:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:26.308421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:26.325965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:26.444622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50647ee-b17a-4df8-b444-dc39157c3d74 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:26.484608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:26.485633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:26.486122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:26.486611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:26.499244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:26.501138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:26.518963 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:26.520046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:26.520990 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:26.520990 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:26.520990 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:26.523280 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 456/903] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:26.532519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:26.550545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:26.656827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367ec01-2bf4-44b2-94d0-5b9c1ead197e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:26.706324 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:26.707127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:26.707452 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:26.707753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:26.708617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:26.708931 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:26.710428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:26.728653 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 448/904] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:26.735194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:26.816992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:26.865474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05d36b-0ab1-4bec-90f2-e96f5a2b7e0a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:26.909777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:26.910433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:26.910775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:26.911089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:26.921636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:26.923253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:26.936886 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:26.937923 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:26.939205 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 457/905] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:46:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:26.946621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:27.041592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:27.077886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10931d66-2d13-4f3d-b898-7a4f99c32607 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:27.124599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:27.125270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:27.125569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:27.125884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:27.126685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:27.126997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:27.128740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:27.156275 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 449/906] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Sep 25 00:46:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:27.164704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:27.284967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9846-4fff-4862-a925-a40cbd3b3153 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:27.323917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:27.328756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:27.330624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:27.331419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:27.331699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:27.410104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:27.412131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:27.519594 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 458/907] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:46:27.529830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:27.713650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4d12b-59e1-4db7-b39b-cb193e13df9d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:27.778993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78a52b-1682-4805-aaca-421472f5d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:27.780060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a52b-1682-4805-aaca-421472f5d48e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:27.780598 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a52b-1682-4805-aaca-421472f5d48e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:27.781087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78a52b-1682-4805-aaca-421472f5d48e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:27.784464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:27.801155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78a52b-1682-4805-aaca-421472f5d48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:27.803819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78a52b-1682-4805-aaca-421472f5d48e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8b78a52b-1682-4805-aaca-421472f5d48e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:27.832400 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:27.833445 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:27.834414 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:27.834414 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:27.834414 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:27.835131 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 450/908] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:27.841323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:27.875015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:27.994831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c039754-e244-4588-b91c-7d07939659b1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:28.060239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:28.061370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.061898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.062376 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.063237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:28.063775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:28.065740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:28.087100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7f5d-f192-4307-aef7-37a80ca24ee6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.087100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7f5d-f192-4307-aef7-37a80ca24ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.087100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c7f5d-f192-4307-aef7-37a80ca24ee6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.088337 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 459/910] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:28.093094 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 451/910] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:28.094544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:28.096907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03573bab-a0c2-4e02-baa8-17ed23309741 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.097206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03573bab-a0c2-4e02-baa8-17ed23309741 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.097517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03573bab-a0c2-4e02-baa8-17ed23309741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.118008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-03573bab-a0c2-4e02-baa8-17ed23309741 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:28.119862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-03573bab-a0c2-4e02-baa8-17ed23309741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:28.202768 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 452/911] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:28.208913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:28.244283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4505a-ae75-4dcb-8a5b-efc2c00a2143 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:28.321014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:28.321680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.322033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.322697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.331246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:28.332882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:28.346363 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:28.348556 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:28.350545 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 460/912] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:28.355131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03573bab-a0c2-4e02-baa8-17ed23309741 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:28.497909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03573bab-a0c2-4e02-baa8-17ed23309741 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:28.525031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:28.528890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:28.575145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:28.577134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.578374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.579453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.581250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:28.582665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:28.586838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:28.625267 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 453/913] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:28.630543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:28.669014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d97ac-528b-4a8a-a8b3-83f36de711f0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:28.728506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:28.729575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.730050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.730514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.731856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:28.734375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:28.754527 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:28.756800 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 25 00:46:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:28.764515 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 461/914] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:28.765898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:28.794774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:28.848460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caceb3-295b-4094-b5bd-fbd792d4522a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:28.897006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3205e6a-c3d1-445a-b166-a36fbe5c42b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:28.898176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205e6a-c3d1-445a-b166-a36fbe5c42b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.898705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205e6a-c3d1-445a-b166-a36fbe5c42b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.899152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3205e6a-c3d1-445a-b166-a36fbe5c42b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.900885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3205e6a-c3d1-445a-b166-a36fbe5c42b9 None admin] RBAC: Adding query filter params (role.id=d044315b993e4258ab44b9e5c0009df6, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=616d85945f3d41839443b011bb48c412) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:28.901201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3205e6a-c3d1-445a-b166-a36fbe5c42b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:28.902939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3205e6a-c3d1-445a-b166-a36fbe5c42b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:28.904239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc6eb-22f8-4fcd-9a2a-d8a402764783 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:28.915990 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 454/915] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:28 2022] GET /identity/v3/role_assignments?user.id=616d85945f3d41839443b011bb48c412&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=d044315b993e4258ab44b9e5c0009df6 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:28.939534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c669ee-e3d5-494c-ad1a-d7f17345cc01 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.939887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c669ee-e3d5-494c-ad1a-d7f17345cc01 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.940104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c669ee-e3d5-494c-ad1a-d7f17345cc01 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.941832 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 455/916] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:28.949416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:28.950396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.950846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.951123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.951859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:28.952169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:28.952461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a4969-9eb5-46ef-b290-2a1a419c471c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:28.952737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a4969-9eb5-46ef-b290-2a1a419c471c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:28.953115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a4969-9eb5-46ef-b290-2a1a419c471c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:28.953772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:28.961721 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 462/917] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:28.971701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:28.993517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-717a4969-9eb5-46ef-b290-2a1a419c471c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:28.997329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-717a4969-9eb5-46ef-b290-2a1a419c471c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:29.073089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:29.080810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:29.116448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5c94d-4d79-401d-88fc-4561b05ebf0f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:29.134023 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 456/918] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:29.135964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598b77-9801-4ce7-9563-aca3c269166d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:29.136250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598b77-9801-4ce7-9563-aca3c269166d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:29.136588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598b77-9801-4ce7-9563-aca3c269166d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:29.144260 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 457/919] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:29.147395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31292ae2-11ec-46ed-b1a7-e6389229ce31 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:29.148108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31292ae2-11ec-46ed-b1a7-e6389229ce31 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:29.148800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31292ae2-11ec-46ed-b1a7-e6389229ce31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:29.172639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:29.173680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:29.174236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:29.174667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:29.176297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:29.176763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:29.178715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:29.187157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-31292ae2-11ec-46ed-b1a7-e6389229ce31 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:29.188607 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 463/920] 10.222.0.26 () {56 vars in 1114 bytes} [Sun Sep 25 00:46:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:29.190099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-31292ae2-11ec-46ed-b1a7-e6389229ce31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:29.197011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:29.320800 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 458/921] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:29.399952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a696461-a59e-4260-b045-d87bdcf30f30 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:29.469655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:29.470784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:29.471185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:29.471578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:29.472622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:29.475069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:29.482695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:29.511350 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 464/922] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:29.614586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31292ae2-11ec-46ed-b1a7-e6389229ce31 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:29.630925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:29.771883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:29.777014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31292ae2-11ec-46ed-b1a7-e6389229ce31 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:29.845085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:29.845800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:29.846412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:29.846729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:29.847721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:29.850490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:29.862408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320810f3-a9f9-4374-803d-70c56330110d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:29.872474 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:29.873842 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:29.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 459/923] 10.222.0.26 () {56 vars in 1132 bytes} [Sun Sep 25 00:46:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:29.884194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:29.924987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:29.926178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:29.926509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:29.926808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:29.927506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:29.929421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:29.946025 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:29.947716 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:29.948636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:29.949403 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 465/924] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:29.956316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:30.023906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:30.079764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c096ff0-e7cf-41e6-abfa-6e394aec96ff None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:30.083555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e80d8-e0fe-4f64-b9fc-05e90120061e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:30.125944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:30.127071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:30.127626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:30.128119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:30.129040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:30.129644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:30.131397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:30.142667 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 466/925] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:30.143985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:30.154966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:30.155638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:30.156115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:30.156616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:30.157758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:30.158266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:30.160890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:30.175833 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 460/926] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:30.177793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:30.230939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:30.284696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92326ca9-a401-42b0-b4ae-39aa897f9f1b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:30.304750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:30.326780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:30.327575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:30.328014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:30.328541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:30.329561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:30.329561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:30.331223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:30.344952 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 467/927] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:30.349799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:30.357530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db14b-acb0-4d4a-bdaf-2f95c2811f7d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:30.415728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:30.416986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:30.417738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:30.418444 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:30.419879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:30.420586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:30.423063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:30.433676 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 461/928] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:30.443970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:30.551997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a985d36-a1ab-473c-8dcc-8dd6e3ed5dc9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:30.608024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:30.608695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:30.609041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:30.609382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:30.610169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] RBAC: Adding query filter params (interface=public, service_id=0c4ef967d85b4e6d8463254cb5bd8e6e, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:30.610548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:30.612098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:30.615586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:30.624509 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 468/929] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:46:30 2022] GET /identity/v3/endpoints?service_id=0c4ef967d85b4e6d8463254cb5bd8e6e&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:30.626211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:30.643485 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f9825-b2ce-4276-87d6-ccb7e37ba566 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:30.687777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:30.710091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cc085-9c2b-4f8a-a4c3-3c2c8865244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:30.711621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cc085-9c2b-4f8a-a4c3-3c2c8865244c None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:30.712459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cc085-9c2b-4f8a-a4c3-3c2c8865244c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:30.713001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cc085-9c2b-4f8a-a4c3-3c2c8865244c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:30.714731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cc085-9c2b-4f8a-a4c3-3c2c8865244c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:30.717249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cc085-9c2b-4f8a-a4c3-3c2c8865244c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:30.750336 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 462/930] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:46:30.776856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3d803-7b69-41d8-a62c-e5607afdeaac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:30.833339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37081813-78e5-4e17-9e84-085eec013991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:30.833947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37081813-78e5-4e17-9e84-085eec013991 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:30.834228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37081813-78e5-4e17-9e84-085eec013991 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:30.834500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37081813-78e5-4e17-9e84-085eec013991 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:30.835301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37081813-78e5-4e17-9e84-085eec013991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:30.837054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37081813-78e5-4e17-9e84-085eec013991 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:30.846835 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 469/931] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:30 2022] GET /identity/v3/services => generated 1497 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 00:46:31.627399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730231a2-a941-4008-8f59-0ee2d8d1ee63 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:31.629023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730231a2-a941-4008-8f59-0ee2d8d1ee63 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:31.629812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730231a2-a941-4008-8f59-0ee2d8d1ee63 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:31.632931 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 463/932] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:31.645640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fff234-15a5-402e-acf8-b55c056b269e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:31.646187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fff234-15a5-402e-acf8-b55c056b269e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:31.646594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fff234-15a5-402e-acf8-b55c056b269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:31.678610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d3fff234-15a5-402e-acf8-b55c056b269e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:31.680721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3fff234-15a5-402e-acf8-b55c056b269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:31.811943 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 470/933] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:32.318988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730231a2-a941-4008-8f59-0ee2d8d1ee63 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:32.483210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730231a2-a941-4008-8f59-0ee2d8d1ee63 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:32.541613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:32.542669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:32.542962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:32.543232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:32.563564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:32.573025 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:32.574122 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:32.575193 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:32.575885 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 464/934] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Sep 25 00:46:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:32.583917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fff234-15a5-402e-acf8-b55c056b269e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:32.625759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:32.726827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fff234-15a5-402e-acf8-b55c056b269e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:32.764413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:32.765060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:32.765596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:32.765871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:32.766549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:32.766853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:32.768311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:32.777754 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 471/935] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Sep 25 00:46:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:32.783599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:32.788691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:32.914229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d43f4-16fd-402c-8f33-cf32cab6af1a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:32.961494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:32.962256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:32.963193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:32.963469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:32.978092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:32.979647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:32.999500 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:33.002049 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:33.004179 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:33.004179 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:33.004179 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:33.004179 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 465/936] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:33.007028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:33.153552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:33.163400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8e17f-0edf-4638-b153-9b8b007561fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:33.213695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fd282-1c1f-4d5a-87a4-6495c9adf91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:33.214405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fd282-1c1f-4d5a-87a4-6495c9adf91a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.214694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fd282-1c1f-4d5a-87a4-6495c9adf91a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.214965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fd282-1c1f-4d5a-87a4-6495c9adf91a None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.215705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fd282-1c1f-4d5a-87a4-6495c9adf91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:33.216200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fd282-1c1f-4d5a-87a4-6495c9adf91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:33.217811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fd282-1c1f-4d5a-87a4-6495c9adf91a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:33.235193 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 472/937] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:33.240914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:33.302934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e30c-aedd-4ab9-922b-97eb90a69726 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.303399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e30c-aedd-4ab9-922b-97eb90a69726 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.304954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e30c-aedd-4ab9-922b-97eb90a69726 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.307938 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 473/938] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:33.316909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39e0f0-c327-42e8-a912-30eb57abdb8c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.317364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39e0f0-c327-42e8-a912-30eb57abdb8c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.317809 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39e0f0-c327-42e8-a912-30eb57abdb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.320152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:33.351103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6f39e0f0-c327-42e8-a912-30eb57abdb8c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:33.353769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6f39e0f0-c327-42e8-a912-30eb57abdb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:33.377625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0ac2b-6445-4980-956d-b97cd7323ac8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:33.433430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:33.434427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.434729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.435001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.446231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:33.448482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:33.465230 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:33.466266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:33.467255 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 466/939] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:46:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:33.473535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:33.477810 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 474/940] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:33.566116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1249d-ebce-42c1-8653-08bb984fdd56 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.566628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1249d-ebce-42c1-8653-08bb984fdd56 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.567570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1249d-ebce-42c1-8653-08bb984fdd56 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.570906 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 475/941] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:33.582691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1da22-b95b-45dd-b041-b44debe37356 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.583315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1da22-b95b-45dd-b041-b44debe37356 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.583800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1da22-b95b-45dd-b041-b44debe37356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.612293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fdd1da22-b95b-45dd-b041-b44debe37356 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:33.614275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fdd1da22-b95b-45dd-b041-b44debe37356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:33.660971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe9adf-3583-4248-83b3-4767097b4910 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:33.706431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:33.724772 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 476/942] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:33.724772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a6a20-3612-4251-8730-f47144a5f36f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.724772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a6a20-3612-4251-8730-f47144a5f36f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.724772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a6a20-3612-4251-8730-f47144a5f36f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.724772 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 477/943] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:33.730996 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526473d-bb09-477c-a5f0-b914bb4a7ad8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.731868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526473d-bb09-477c-a5f0-b914bb4a7ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.731868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526473d-bb09-477c-a5f0-b914bb4a7ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.751427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:33.752771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.753395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.753887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.755474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:33.756063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:33.758616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:33.760352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6526473d-bb09-477c-a5f0-b914bb4a7ad8 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:33.763214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6526473d-bb09-477c-a5f0-b914bb4a7ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:33.782073 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 467/944] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Sep 25 00:46:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:33.783250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:33.794649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:33.905255 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 478/945] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:33.906867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526473d-bb09-477c-a5f0-b914bb4a7ad8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:33.938080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e158d8-df9b-4e9f-84cc-a57126336c1c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:33.993357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:33.995232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:33.995972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:33.996612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:33.997872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.000897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.005880 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING keystone.server.flask.common [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 25 00:46:34.037645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 25 00:46:34.037645 dv-cinder-master-3-smb devstack@keystone.service[81500]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 25 00:46:34.037645 dv-cinder-master-3-smb devstack@keystone.service[81500]: [parameters: {'id': '523093e85efc48958bf1899ff82802cf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 25 00:46:34.037645 dv-cinder-master-3-smb devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 523093e85efc48958bf1899ff82802cf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 523093e85efc48958bf1899ff82802cf. Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 25 00:46:34.055829 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 25 00:46:34.057370 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Sep 25 00:46:34.058467 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 25 00:46:34.059427 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 25 00:46:34.062125 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 25 00:46:34.064145 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'id': '523093e85efc48958bf1899ff82802cf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.067648 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:34.069647 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 523093e85efc48958bf1899ff82802cf. Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 468/946] 10.222.0.26 () {60 vars in 1140 bytes} [Sun Sep 25 00:46:33 2022] POST /identity/v3/projects => generated 262 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:34.072471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.104397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526473d-bb09-477c-a5f0-b914bb4a7ad8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.115085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:34.168855 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.169903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.170386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.170879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.189808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.192566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.208512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:34.221062 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:34.228625 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:34.229613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:34.229613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:34.229613 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:34.229613 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 479/947] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:34.229613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.230659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07bd18-f76a-43b4-be14-cf4fe8f9a9ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.301904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.303027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.303529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.304078 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.307163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.307757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:34.321150 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:34.322804 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.323652 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 469/948] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:34.324665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.385285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fe7a6-11fc-47d5-a53d-f83841cc421d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.424814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.425594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.425879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.426157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.434310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.435992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.447901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:34.451756 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:34.453196 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:34.454483 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:34.460285 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 480/949] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:34.460722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92845f94-03c5-45bc-9649-56bb67613f76 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.461325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.523551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.524335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.524758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.525177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.526029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.527810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:34.538001 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:34.540292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:34.543165 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 470/950] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:34.545216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.546661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:34.610436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11b7af-4e2f-4a7f-b292-d9564db4b552 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.673591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.674571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.675036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.675492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.678628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:34.679146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.684483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.703772 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 481/951] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:34.709334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.710378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c28a2-483e-475e-8651-055113059b62 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.755785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.756057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.756194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.756388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.757053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:34.757357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.758729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.767194 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 471/952] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:34.768025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.796207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:34.841172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347993-ccf5-442b-88af-51bbf449e483 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.873632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:34.892552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.893330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.893736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.894125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.894922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:34.895318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.897036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.901609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c1714-4210-45f1-9a22-975a282fe36a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:34.917437 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 482/953] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:34.918858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:34.945394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:34.946205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:34.946399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:34.946693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:34.947433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:34.947741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:34.949182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:34.962994 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 472/954] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:34.962994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:35.117375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a2908-fd4e-4b23-8c68-6822ffd96ebb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:35.122237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:35.150703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf201a-641b-457a-9713-799f4b90469b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:35.175441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:35.176105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:35.176546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:35.176901 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:35.186544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:35.188208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:35.202388 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:35.203385 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 483/955] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:35.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:35.211256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:35.212172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:35.212662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:35.213115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:35.214203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:35.214623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:35.216591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:35.225280 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 473/956] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:35.226698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:35.285812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:35.392291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66fa7-b515-4f74-b056-22affa58eb20 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:35.436510 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e5808-5357-4915-9ddd-ba99f91cb0dc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:35.465658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:35.466293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:35.466635 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:35.466917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:35.467579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:35.467887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:35.469369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:35.475834 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 484/957] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:35.482265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:35.536221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:35.537809 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:35.538542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:35.539063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:35.540422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:35.541664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:35.544318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:35.566785 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 474/958] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:35.575838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:35.604681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:35.654471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea27b-8cb1-4362-a21e-e937e509f6c0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:35.657496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:35.704140 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600073b-9da5-4c99-94ce-8540fb1b9ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:35.704924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600073b-9da5-4c99-94ce-8540fb1b9ad3 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:35.705311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600073b-9da5-4c99-94ce-8540fb1b9ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:35.705776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600073b-9da5-4c99-94ce-8540fb1b9ad3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:35.706644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600073b-9da5-4c99-94ce-8540fb1b9ad3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:35.708164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600073b-9da5-4c99-94ce-8540fb1b9ad3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:35.734906 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 485/959] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:46:35.780543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f31504-d6d3-4181-9cd5-7264dd8a90ca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:35.837017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ed489-d2c1-47b7-bbd1-24810885ef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:35.838412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ed489-d2c1-47b7-bbd1-24810885ef7b None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:35.838709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ed489-d2c1-47b7-bbd1-24810885ef7b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:35.839125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ed489-d2c1-47b7-bbd1-24810885ef7b None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/616d85945f3d41839443b011bb48c412/roles/d044315b993e4258ab44b9e5c0009df6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:35.841752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ed489-d2c1-47b7-bbd1-24810885ef7b None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=616d85945f3d41839443b011bb48c412, role_id=d044315b993e4258ab44b9e5c0009df6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:35.843418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ed489-d2c1-47b7-bbd1-24810885ef7b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:35.863791 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 475/960] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:46:35 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/616d85945f3d41839443b011bb48c412/roles/d044315b993e4258ab44b9e5c0009df6 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:46:37.808653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8b8fb-8622-4154-add2-50872d5b0ebb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:37.809394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8b8fb-8622-4154-add2-50872d5b0ebb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:37.809535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8b8fb-8622-4154-add2-50872d5b0ebb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:37.811850 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 486/961] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:37.819218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023e2f-97ac-45ef-a3e0-d05876b7a70f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:37.819727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023e2f-97ac-45ef-a3e0-d05876b7a70f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:37.819804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023e2f-97ac-45ef-a3e0-d05876b7a70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:37.847424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fd023e2f-97ac-45ef-a3e0-d05876b7a70f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:37.849108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fd023e2f-97ac-45ef-a3e0-d05876b7a70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:37.959578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:37.972710 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 476/962] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:38.252851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8b8fb-8622-4154-add2-50872d5b0ebb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:38.380313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8b8fb-8622-4154-add2-50872d5b0ebb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:38.415771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd683bb-8906-4823-bb80-5c714d542775 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.416468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd683bb-8906-4823-bb80-5c714d542775 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.416965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd683bb-8906-4823-bb80-5c714d542775 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.418950 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 477/963] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:38.423874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:38.424700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.424985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.425295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.425976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:38.426419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1857-bc51-4b94-a8a5-a4be5cb72fcb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.426919 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1857-bc51-4b94-a8a5-a4be5cb72fcb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.427450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:38.427677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1857-bc51-4b94-a8a5-a4be5cb72fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.430250 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING keystone.server.flask.common [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 25 00:46:38.444200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.sql.core [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 25 00:46:38.444200 dv-cinder-master-3-smb devstack@keystone.service[81499]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 25 00:46:38.444200 dv-cinder-master-3-smb devstack@keystone.service[81499]: [parameters: {'user_id': '2d5ca295511748588cb3c8ad32d09cf7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 25 00:46:38.444200 dv-cinder-master-3-smb devstack@keystone.service[81499]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81499) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 25 00:46:38.450570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-315e1857-bc51-4b94-a8a5-a4be5cb72fcb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:38.452133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-315e1857-bc51-4b94-a8a5-a4be5cb72fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application result = self._query(query) Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application conn.query(q) Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application result.read() Sep 25 00:46:38.467891 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 25 00:46:38.469158 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application next(self.gen) Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application yield resource Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application next(self.gen) Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.session.rollback() Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.force_reraise() Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise self.value Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application session.commit() Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 25 00:46:38.470154 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._prepare_impl() Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.session.flush() Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._flush(objects) Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application compat.raise_( Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application flush_context.execute() Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rec.execute(self) Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 25 00:46:38.471064 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 25 00:46:38.472818 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application result = self._query(query) Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application conn.query(q) Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application result.read() Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 25 00:46:38.473794 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [parameters: {'user_id': '2d5ca295511748588cb3c8ad32d09cf7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:38.474653 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 25 00:46:38.475626 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 25 00:46:38.476568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 25 00:46:38.476568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 25 00:46:38.476568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 25 00:46:38.476568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 25 00:46:38.476568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 25 00:46:38.476568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:38.476568 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 487/964] 10.222.0.26 () {60 vars in 1131 bytes} [Sun Sep 25 00:46:38 2022] POST /identity/v3/users => generated 160 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:38.478652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:38.524475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:38.585700 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 478/965] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:38.588280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592124-5c27-4c29-934c-e0076619f624 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.588825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592124-5c27-4c29-934c-e0076619f624 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.589313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91592124-5c27-4c29-934c-e0076619f624 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.596519 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 479/966] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:38.598917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539938b-9787-492e-8134-3653d6c5de96 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.599420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539938b-9787-492e-8134-3653d6c5de96 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.599923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539938b-9787-492e-8134-3653d6c5de96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.644022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2539938b-9787-492e-8134-3653d6c5de96 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:38.653302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2539938b-9787-492e-8134-3653d6c5de96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:38.657099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b182d-29fd-4e20-98d0-24792973680e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:38.708767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:38.709845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.710510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.711010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.726534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:38.729768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:38.745726 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:38.746777 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:38.747762 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 488/967] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:46:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:38.756868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:38.784664 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 480/968] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:38.914594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:38.929099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc61af-a5b1-4987-acab-3fc6215ce614 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.929654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc61af-a5b1-4987-acab-3fc6215ce614 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.930112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc61af-a5b1-4987-acab-3fc6215ce614 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.937087 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 481/969] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:38.943358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f0c9e-6f21-49be-afeb-3d86e852dbde None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:38.943868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f0c9e-6f21-49be-afeb-3d86e852dbde None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:38.944308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f0c9e-6f21-49be-afeb-3d86e852dbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:38.951286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb173bbd-d100-47d1-91d5-562da29b5ef7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:38.981769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-713f0c9e-6f21-49be-afeb-3d86e852dbde None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:38.984812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-713f0c9e-6f21-49be-afeb-3d86e852dbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:39.007730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:39.008698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:39.009495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:39.010110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:39.011628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:39.012209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:39.014827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:39.087014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:39.092627 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 489/970] 10.222.0.26 () {56 vars in 1101 bytes} [Sun Sep 25 00:46:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:39.094130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:39.124076 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 482/971] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:39.125453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f0c9e-6f21-49be-afeb-3d86e852dbde None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:39.314406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8047ed4-5677-48e6-a768-2e2142bdf29f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:39.363589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f0c9e-6f21-49be-afeb-3d86e852dbde None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:39.372729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:39.373406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:39.373695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:39.373969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:39.374642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:39.376421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:39.394691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:39.411851 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 490/972] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:46:39 2022] POST /identity/v3/services => generated 242 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:39.444936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:39.446160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:39.446633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:39.447675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:39.448509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:39.450195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:39.457814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:39.465998 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:39.467196 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:39.468063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:39.470431 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 483/973] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:39.478383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:39.544899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:39.635946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d354-632b-4e05-8e59-fe2f15570525 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:39.640346 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195589ba-ff01-482e-823b-a0b765054f14 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:39.685064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:39.686079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:39.686079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:39.686270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:39.686985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:39.687270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:39.688607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:39.695972 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 484/974] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:39.702136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:39.705197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:39.706225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:39.706699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:39.707161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:39.727407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:39.737807 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:39.739831 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:39.741869 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 491/975] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Sep 25 00:46:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:39.746942 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:39.850293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58610fd0-8024-4b65-a5c8-e399c640cfb6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:39.863288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:39.863750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:39.893812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:39.894349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:39.894630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:39.894902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:39.895546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:39.895848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:39.897278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:39.908203 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 485/976] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:39.909921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:39.940699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119ead7-6308-473e-9cd8-b8dfa7e2c8b4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:39.999089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:39.999714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:39.999997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:40.000272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:40.001005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:40.001364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:40.002772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:40.014304 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 492/977] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Sep 25 00:46:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:40.020643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:40.038042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736638a-880d-40db-a6b9-d445d02f6fb4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:40.083789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:40.084864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:40.085331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:40.085806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:40.087139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] RBAC: Adding query filter params (interface=admin, service_id=0c4ef967d85b4e6d8463254cb5bd8e6e, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:40.087707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:40.090301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:40.108885 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 486/978] 10.222.0.26 () {56 vars in 1249 bytes} [Sun Sep 25 00:46:39 2022] GET /identity/v3/endpoints?service_id=0c4ef967d85b4e6d8463254cb5bd8e6e&interface=admin®ion_id=RegionOne => generated 193 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:40.110681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:40.191503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:40.193019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6bbd8d-237e-4583-9397-d0bc581a3b87 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:40.225484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:40.242788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:40.243590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:40.244449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:40.244740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:40.259980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:40.261864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:40.275848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d714b0-e165-42c9-a1b0-d1d24c708677 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:40.282751 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:40.284656 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:40.286620 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:40.286620 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:40.286620 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:40.286620 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 493/979] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:40.293035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:40.344058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:40.344805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:40.345425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:40.345713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:40.346465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:40.347886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:40.359349 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 487/980] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:40 2022] GET /identity/v3/services => generated 1727 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 00:46:40.435183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736a612-220b-4124-a738-24a63bf55185 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:40.480225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:40.480225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:40.480535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:40.480843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:40.481514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:40.482024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:40.483298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:40.498150 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 494/981] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:40.504279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:40.631560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813266e-9e77-4017-8638-2e4aae5e3c18 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:40.633289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:40.675116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:40.675770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:40.676091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:40.676304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:40.686159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:40.687621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:40.700144 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:40.701506 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:40.702512 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 488/982] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:46:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:40.709499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:40.766928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:40.906980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699012b9-fec0-4841-b430-63dff42bba99 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:40.966186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:40.966874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:40.967134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:40.967364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:40.968117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:40.968456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:40.969944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:40.990008 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 495/983] 10.222.0.26 () {56 vars in 1149 bytes} [Sun Sep 25 00:46:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:40.996021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:41.102904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde0ce-0188-44fc-9d33-ceb1f29456c6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:41.138632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:41.139244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:41.139521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:41.139794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:41.172053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:41.213202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:41.214669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:41.362749 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 489/984] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:40 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:46:41.369977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:41.476778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:41.491157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae154ea-0265-4e41-91eb-3937e9d7bbb7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:41.550470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:41.551559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:41.552053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:41.552548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:41.572011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:41.574749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:41.601834 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:41.604298 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:41.606266 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:41.606266 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:41.606266 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:41.606266 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 496/985] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:41.616183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:41.711767 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:41.767212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dd440-017e-4fcc-bd4e-282d562c24fb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:41.815599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:41.816946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:41.817554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:41.817989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:41.818822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:41.819516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:41.821523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:41.840987 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 490/986] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:41.851025 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:42.026823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc26d59-ce04-4355-843b-2f8a9b1a19d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:42.068174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:42.068938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:42.069224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:42.069587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:42.079613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:42.081326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:42.105865 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:42.107046 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:42.107985 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:42.113681 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 497/987] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:42.114542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:42.233614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ae829-c747-47cd-9f07-546ec3405197 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:42.259136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:42.273963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f454c-0eed-4db7-92b1-804e7ed56676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:42.274581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f454c-0eed-4db7-92b1-804e7ed56676 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:42.275437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f454c-0eed-4db7-92b1-804e7ed56676 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:42.275720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f454c-0eed-4db7-92b1-804e7ed56676 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:42.276371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f454c-0eed-4db7-92b1-804e7ed56676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:42.276711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f454c-0eed-4db7-92b1-804e7ed56676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:42.278231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f454c-0eed-4db7-92b1-804e7ed56676 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:42.293229 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 491/988] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:42.299145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:42.437896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65696dd-3061-46a8-869a-238668c27ac3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:42.476133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babaa97-887f-469d-b637-9a6cc3b3937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:42.476843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babaa97-887f-469d-b637-9a6cc3b3937a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:42.477122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babaa97-887f-469d-b637-9a6cc3b3937a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:42.477388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babaa97-887f-469d-b637-9a6cc3b3937a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:42.478164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babaa97-887f-469d-b637-9a6cc3b3937a None admin] RBAC: Adding query filter params (role.id=d044315b993e4258ab44b9e5c0009df6, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=616d85945f3d41839443b011bb48c412) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:42.478471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babaa97-887f-469d-b637-9a6cc3b3937a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:42.479928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babaa97-887f-469d-b637-9a6cc3b3937a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:42.487207 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 498/989] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:42 2022] GET /identity/v3/role_assignments?user.id=616d85945f3d41839443b011bb48c412&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=d044315b993e4258ab44b9e5c0009df6 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:43.002647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455ef74-0b34-4736-882c-6f3de2aecd03 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:43.003178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455ef74-0b34-4736-882c-6f3de2aecd03 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:43.003614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455ef74-0b34-4736-882c-6f3de2aecd03 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:43.006487 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 492/990] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:43.016832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e57bd-957b-4ec7-a04c-723666b74dd2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:43.017373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e57bd-957b-4ec7-a04c-723666b74dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:43.017786 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e57bd-957b-4ec7-a04c-723666b74dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:43.049132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-204e57bd-957b-4ec7-a04c-723666b74dd2 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:43.051834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-204e57bd-957b-4ec7-a04c-723666b74dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:43.178229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:43.189701 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 499/991] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:43.685644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455ef74-0b34-4736-882c-6f3de2aecd03 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:43.858238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455ef74-0b34-4736-882c-6f3de2aecd03 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:43.911310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cbcf8-d264-4ceb-8902-83c946b1829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:43.911964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbcf8-d264-4ceb-8902-83c946b1829d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:43.912246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbcf8-d264-4ceb-8902-83c946b1829d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:43.912545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbcf8-d264-4ceb-8902-83c946b1829d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:43.913132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cbcf8-d264-4ceb-8902-83c946b1829d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:43.914496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cbcf8-d264-4ceb-8902-83c946b1829d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0a1cbcf8-d264-4ceb-8902-83c946b1829d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:43.923981 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:43.925339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:43.926218 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 493/992] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:43.935248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e57bd-957b-4ec7-a04c-723666b74dd2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:44.062970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f350c3-061e-4334-b89d-d62bf601da04 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.063409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f350c3-061e-4334-b89d-d62bf601da04 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.063801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f350c3-061e-4334-b89d-d62bf601da04 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.065708 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 494/993] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:44.071998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3982407-2fce-4de2-8c93-6e278bf2427e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.072959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3982407-2fce-4de2-8c93-6e278bf2427e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.073362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3982407-2fce-4de2-8c93-6e278bf2427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.095228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e3982407-2fce-4de2-8c93-6e278bf2427e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:44.096646 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e3982407-2fce-4de2-8c93-6e278bf2427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:44.109430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e57bd-957b-4ec7-a04c-723666b74dd2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:44.150360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:44.150982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.151263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.151531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.152172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:44.152508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:44.154004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:44.156490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:44.166173 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 500/994] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:44.167073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:44.190972 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 495/995] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:44.255970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:44.324632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c86be3-ad5e-4af2-8c81-73674cdd979d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:44.401218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:44.401854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.402137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.402411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.403072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:44.403445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:44.404926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:44.411620 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 501/996] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:44.418933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3982407-2fce-4de2-8c93-6e278bf2427e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:44.469835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:44.538720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3982407-2fce-4de2-8c93-6e278bf2427e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:44.576859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:44.577389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.578391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.578391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.578803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] RBAC: Adding query filter params (interface=public, service_id=5880378de0c449358c66ace271dcb3d2, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:44.579113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:44.580564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:44.591656 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 496/997] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:46:44 2022] GET /identity/v3/endpoints?service_id=5880378de0c449358c66ace271dcb3d2&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:44.593205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:44.596181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a0e47-2682-46cb-81ef-bdd7ebb92ec4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:44.631003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:44.639793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:44.644016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:44.644917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.645356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.645805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.646566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:44.648122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:44.658532 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:44.659885 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 25 00:46:44.660971 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:44.664436 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 502/998] 10.222.0.26 () {56 vars in 1126 bytes} [Sun Sep 25 00:46:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:44.671225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:44.731904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f880c00-f114-4416-ad04-03fcf4674882 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:44.777372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:44.777989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.778277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.778551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.779173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:44.780579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:44.787504 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 497/999] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:44 2022] GET /identity/v3/services => generated 1727 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 00:46:44.839555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38657f11-9b8f-4f6b-80d7-bf8e71ae4e5e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:44.907592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b2c2-6885-4875-96d1-661c03324c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:44.908629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b2c2-6885-4875-96d1-661c03324c19 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.909137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b2c2-6885-4875-96d1-661c03324c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.909684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b2c2-6885-4875-96d1-661c03324c19 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.910537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b2c2-6885-4875-96d1-661c03324c19 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:44.911080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b2c2-6885-4875-96d1-661c03324c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:44.913765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b2c2-6885-4875-96d1-661c03324c19 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:44.928472 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 503/1000] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:44.929678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:44.974477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0a342-76c7-4b59-af29-6844258c1d26 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.974817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0a342-76c7-4b59-af29-6844258c1d26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.975034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0a342-76c7-4b59-af29-6844258c1d26 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:44.976618 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 504/1001] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:44.982984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e4a53-ee45-48a4-ad50-64e3afe43ce1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:44.983326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e4a53-ee45-48a4-ad50-64e3afe43ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:44.983545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e4a53-ee45-48a4-ad50-64e3afe43ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:45.003712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-805e4a53-ee45-48a4-ad50-64e3afe43ce1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:45.005039 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-805e4a53-ee45-48a4-ad50-64e3afe43ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:45.080715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:45.083668 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949d26a-b91b-4dbd-a3df-df0f6249b558 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:45.088957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:45.104656 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 505/1002] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:45.149249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:45.150324 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:45.150621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:45.150889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:45.151644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:45.152189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:45.154882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:45.164415 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 498/1003] 10.222.0.26 () {56 vars in 1122 bytes} [Sun Sep 25 00:46:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:45.174095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e4a53-ee45-48a4-ad50-64e3afe43ce1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:45.320541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e4a53-ee45-48a4-ad50-64e3afe43ce1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:45.387128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:45.388102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:45.388699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:45.389131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:45.390343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:45.393170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:45.414894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:45.419691 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 506/1004] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:46:45.588646 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f728c5-011f-4dc0-bac1-92f8904c25fa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:45.661650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:45.663002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:45.663002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:45.663313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] PATH_INFO: `/v3/services/image` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:45.664219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:45.667160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:45.680812 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:45.683646 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:45.684984 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 499/1005] 10.222.0.26 () {56 vars in 1105 bytes} [Sun Sep 25 00:46:45 2022] GET /identity/v3/services/image => generated 86 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:45.691601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:45.772989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:45.862028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825d9ce-a46b-4dab-835f-042a057942d5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:45.907104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572545-e22d-4971-9f12-4da1d00683ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:45.907757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572545-e22d-4971-9f12-4da1d00683ab None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:45.908431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572545-e22d-4971-9f12-4da1d00683ab None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:45.908719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572545-e22d-4971-9f12-4da1d00683ab None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:45.909564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572545-e22d-4971-9f12-4da1d00683ab None admin] RBAC: Adding query filter params (name=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:45.909999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572545-e22d-4971-9f12-4da1d00683ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:45.912041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70572545-e22d-4971-9f12-4da1d00683ab None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:45.927258 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 507/1006] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:45.930302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:45.970069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:46.090693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a6486-c2a5-4daa-98e2-aba201319d31 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:46.157039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951995c8-4eb2-4fa6-a4a5-a8fbf074491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:46.158065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951995c8-4eb2-4fa6-a4a5-a8fbf074491c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:46.158544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951995c8-4eb2-4fa6-a4a5-a8fbf074491c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:46.159005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951995c8-4eb2-4fa6-a4a5-a8fbf074491c None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:46.160077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951995c8-4eb2-4fa6-a4a5-a8fbf074491c None admin] RBAC: Adding query filter params (type=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:46.160629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951995c8-4eb2-4fa6-a4a5-a8fbf074491c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:46.163906 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951995c8-4eb2-4fa6-a4a5-a8fbf074491c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:46.173736 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 500/1007] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:47.207860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba10e55-6a7d-4866-9765-6c1714d55ff6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:47.208738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba10e55-6a7d-4866-9765-6c1714d55ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:47.208923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba10e55-6a7d-4866-9765-6c1714d55ff6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:47.210872 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 508/1008] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:47.217728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9248b0-673f-48ed-80ff-b794c9c9c85e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:47.218578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9248b0-673f-48ed-80ff-b794c9c9c85e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:47.218733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9248b0-673f-48ed-80ff-b794c9c9c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:47.239763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8f9248b0-673f-48ed-80ff-b794c9c9c85e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:47.241328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8f9248b0-673f-48ed-80ff-b794c9c9c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:47.385145 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 501/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:47.877632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba10e55-6a7d-4866-9765-6c1714d55ff6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:48.064056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba10e55-6a7d-4866-9765-6c1714d55ff6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:48.074964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:48.083337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8d5d0-fdab-4c07-b309-4c9dfe8506a7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.083643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8d5d0-fdab-4c07-b309-4c9dfe8506a7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.083881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8d5d0-fdab-4c07-b309-4c9dfe8506a7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.085539 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 502/1010] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:48.092002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b81e79-88b8-4059-8111-a85acf514617 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.092418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b81e79-88b8-4059-8111-a85acf514617 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.092796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b81e79-88b8-4059-8111-a85acf514617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.112187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:48.113053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.113492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.113919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.114641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:48.115866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c5b81e79-88b8-4059-8111-a85acf514617 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:48.116420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:48.117223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c5b81e79-88b8-4059-8111-a85acf514617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:48.128322 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:48.130312 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:48.131434 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 509/1011] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:48.137260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:48.146569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:48.213217 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 503/1012] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:48.264845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9396e-ea7b-45f2-b964-a12948f1153a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:48.314287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:48.314918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.315199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.315487 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.316104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:48.316421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:48.317959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:48.324752 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 510/1013] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:48.330745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b81e79-88b8-4059-8111-a85acf514617 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:48.453827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b81e79-88b8-4059-8111-a85acf514617 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:48.490085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:48.496149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:48.496976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.497272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.497541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.498177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:48.498604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:48.499892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:48.505891 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 504/1014] 10.222.0.26 () {56 vars in 1112 bytes} [Sun Sep 25 00:46:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:48.512146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:48.536722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:48.543154 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:48.622160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775306f-e943-4a95-b4d1-337a72c1cc68 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:48.659612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:48.660629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.661087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.661536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.663390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:48.665125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:48.667882 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 511/1015] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:48.673605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:48.687816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e11413-bdfe-4be7-88b2-7955f8ba1294 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:48.743097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938e43e-1d7c-457e-8a17-60abeb706018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:48.744145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e43e-1d7c-457e-8a17-60abeb706018 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.744696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e43e-1d7c-457e-8a17-60abeb706018 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.745200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938e43e-1d7c-457e-8a17-60abeb706018 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.746447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938e43e-1d7c-457e-8a17-60abeb706018 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:48.749083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938e43e-1d7c-457e-8a17-60abeb706018 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:48.781928 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 505/1016] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:48.813132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c519a7-3727-4003-a89f-fce1813238e5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:48.816319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:48.819165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405ae6f-c467-40ea-89cb-bceeb1455060 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.819711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405ae6f-c467-40ea-89cb-bceeb1455060 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.820231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405ae6f-c467-40ea-89cb-bceeb1455060 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.823571 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 506/1017] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:48.831680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431058c-cbbf-432a-bc10-d3c2c5117637 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.832186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431058c-cbbf-432a-bc10-d3c2c5117637 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.832733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431058c-cbbf-432a-bc10-d3c2c5117637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.865545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:48.865875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f431058c-cbbf-432a-bc10-d3c2c5117637 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:48.866242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:48.866505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:48.866774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:48.867445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:48.867801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f431058c-cbbf-432a-bc10-d3c2c5117637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:48.869099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:48.908600 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 512/1018] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:46:48 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:46:48.973533 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 507/1019] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:48 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:49.297696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:49.425032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d770e-7700-48d5-be9c-6cfdb43a88fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:49.478373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a414a7-0414-4ec8-85b0-5b5f490494b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:49.479019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a414a7-0414-4ec8-85b0-5b5f490494b4 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:49.479301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a414a7-0414-4ec8-85b0-5b5f490494b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:49.479625 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a414a7-0414-4ec8-85b0-5b5f490494b4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:49.480308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a414a7-0414-4ec8-85b0-5b5f490494b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:49.481783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a414a7-0414-4ec8-85b0-5b5f490494b4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:49.500446 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 513/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:46:49 2022] POST /identity/v3/services => generated 242 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:51.519461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a790e-e5b2-4b75-823b-61ed123c29c5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:51.520007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a790e-e5b2-4b75-823b-61ed123c29c5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:51.520007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a790e-e5b2-4b75-823b-61ed123c29c5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:51.521574 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 508/1021] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:51.534629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46a0b3-d8b0-4cb7-82d2-1578647bf91d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:51.535325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46a0b3-d8b0-4cb7-82d2-1578647bf91d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:51.535479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46a0b3-d8b0-4cb7-82d2-1578647bf91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:51.560134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5c46a0b3-d8b0-4cb7-82d2-1578647bf91d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:51.561611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5c46a0b3-d8b0-4cb7-82d2-1578647bf91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:51.569630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c0117-fe16-4d4a-b808-663d4cbb0594 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:51.570105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c0117-fe16-4d4a-b808-663d4cbb0594 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:51.570242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c0117-fe16-4d4a-b808-663d4cbb0594 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:51.572703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:51.574129 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 509/1022] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:51.581603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc23f71-bc4e-49a9-98a4-42c27b4b0a0b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:51.581886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc23f71-bc4e-49a9-98a4-42c27b4b0a0b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:51.582207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc23f71-bc4e-49a9-98a4-42c27b4b0a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:51.589209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:51.604840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5bc23f71-bc4e-49a9-98a4-42c27b4b0a0b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:51.606082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5bc23f71-bc4e-49a9-98a4-42c27b4b0a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:51.692617 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 514/1023] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:51.696927 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 510/1024] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:51.888885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952dae7-fda4-4938-b279-a0658a01296d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:51.889072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952dae7-fda4-4938-b279-a0658a01296d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:51.889423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952dae7-fda4-4938-b279-a0658a01296d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:51.890706 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 515/1025] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:51.896715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f19c7e-bb9f-4123-bd50-9ccd24a15e8f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:51.897069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f19c7e-bb9f-4123-bd50-9ccd24a15e8f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:51.897426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f19c7e-bb9f-4123-bd50-9ccd24a15e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:51.924506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e4f19c7e-bb9f-4123-bd50-9ccd24a15e8f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:51.926357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e4f19c7e-bb9f-4123-bd50-9ccd24a15e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:51.996509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952dae7-fda4-4938-b279-a0658a01296d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.016578 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 511/1026] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:52.131877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952dae7-fda4-4938-b279-a0658a01296d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.187395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:52.187963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:52.188248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:52.188554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:52.196291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:52.202222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:52.207176 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:52.208426 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:52.209568 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:52.213263 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 516/1027] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Sep 25 00:46:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:52.213394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f19c7e-bb9f-4123-bd50-9ccd24a15e8f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.216419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.301475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:52.350594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedca407-1c71-430d-8c0a-ac6a535ca9ac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.357527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f19c7e-bb9f-4123-bd50-9ccd24a15e8f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.388044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:52.388044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:52.388044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:52.389539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:52.389872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:52.390290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:52.392657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:52.398952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:52.399608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:52.399884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:52.400290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:52.401463 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:52.402928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:52.404280 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 517/1028] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Sep 25 00:46:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:52.405197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.408022 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 512/1029] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:52.413631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.486339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:52.546105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648a96f-b315-4778-b39c-3bd38af317a3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.550574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94d862-b6e1-47ef-9468-addebeb15548 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.587017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:52.587876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:52.588162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:52.588636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:52.588806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:52.589648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:52.591624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:52.595599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:52.596256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:52.596566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:52.596841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:52.602473 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:52.604748 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:52.606428 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 518/1030] 10.222.0.26 () {56 vars in 1105 bytes} [Sun Sep 25 00:46:52 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:52.607727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.614030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:52.615584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:52.631942 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:52.633204 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:52.634241 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:52.634241 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:52.634241 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:52.634241 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 513/1031] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:52.634241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.734853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480426d-0d79-4bb1-be70-a2aba06bc0e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.769265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4b79c-8bd7-4d83-a7cc-b0c524c9cb8c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.770183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:52.790458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:52.791510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:52.792003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:52.792557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:52.793830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:52.796631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:52.806850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:52.807467 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:52.807743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:52.808009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:52.808682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] RBAC: Adding query filter params (name=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:52.808982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:52.810391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:52.818293 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 514/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:52.819752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.833280 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 519/1033] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:46:52 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:52.833280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:52.897927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:52.987654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1551961-54de-4580-acd2-8f7d5441b573 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:52.991820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b541c1-f51e-4e0a-ae72-cecebbc1e78a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:53.052157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:53.053111 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:53.053557 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:53.054007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:53.055011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:53.055370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:53.058648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:53.061388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:53.062248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:53.062695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:53.063800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:53.064114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] RBAC: Adding query filter params (type=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:53.064666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:53.067918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:53.075549 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 520/1034] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:53.081647 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 515/1035] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:53.083072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:53.090590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:53.147546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:53.233807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0e42c-18ba-49e5-a667-ce517d3815a2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:53.239068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ba884-e271-42d2-8fb0-fede03f89252 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:53.241984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:53.287224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:53.288008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:53.288515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:53.288925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:53.289767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:53.290347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:53.291171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:53.291450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:53.292451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] RBAC: Adding query filter params (interface=public, service_id=d2edc2ebf4b7416bbcbb24acd924959f, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:53.292643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:53.294285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:53.302450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:53.305182 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 521/1036] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:46:53 2022] GET /identity/v3/endpoints?service_id=d2edc2ebf4b7416bbcbb24acd924959f&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:53.305374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:53.309016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:53.325051 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:53.326168 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:46:53.327173 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:53.328099 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 516/1037] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:46:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:53.333779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:53.477241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37f1fe-2403-4fa7-9a6b-afccf5810e91 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:53.490540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:53.517759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5601ac2-e950-4fe3-935b-04c2feb32a4e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:53.537816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:53.538534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:53.538821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:53.539094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:53.539711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:53.541793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:53.552422 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 522/1038] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:46:53 2022] GET /identity/v3/services => generated 1957 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 00:46:53.574897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:53.575447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:53.575728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:53.576007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:53.577093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:53.577521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:53.579975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:53.612405 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 517/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:46:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:53.614370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:53.753995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449fe82-c28b-4de0-8a5c-64f8ec0c19fa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:53.812407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:53.813248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:53.813715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:53.814190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:53.885598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:53.937646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:53.939964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:54.065593 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 523/1040] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:46:54.067289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:54.094622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:54.184335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f351cf-c2d7-4d05-81fe-4b63ee4adce3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:54.219271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:54.219899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:54.220178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:54.220478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:54.230858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:54.232364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:54.247697 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:54.249048 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:54.250123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:54.250123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:54.250123 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:54.252471 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 518/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:54.254916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:54.345923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:54.372102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7565298-650a-441c-9d90-6034ba9ed456 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:54.412195 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:54.412841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:54.413119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:54.413390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:54.414036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:54.414336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:54.415823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:54.429362 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 524/1042] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:54.436625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:54.546806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:54.574869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4938bc5-a82f-4e00-a49d-0ae5267324be None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:54.613969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:54.614656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:54.614921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:54.615121 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:54.622799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:54.624231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:46:54.635664 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:54.642192 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:54.643135 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 519/1043] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:54.645488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:54.754845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410636-689b-48eb-934b-88eaf6a32483 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:54.787449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:54.795307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe1133-6696-4af4-bf71-b240996eff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:54.795925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1133-6696-4af4-bf71-b240996eff02 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:54.796208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1133-6696-4af4-bf71-b240996eff02 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:54.796514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1133-6696-4af4-bf71-b240996eff02 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:54.797216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1133-6696-4af4-bf71-b240996eff02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:54.797520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1133-6696-4af4-bf71-b240996eff02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:54.798984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1133-6696-4af4-bf71-b240996eff02 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:54.816980 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 525/1044] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:54.818480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:55.005206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03625dd3-275c-45fb-b35a-9dfeb425f190 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:55.063693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48364a-f7ec-43c1-932d-bacdf69551e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:55.064719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48364a-f7ec-43c1-932d-bacdf69551e2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:55.065193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48364a-f7ec-43c1-932d-bacdf69551e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:55.065655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48364a-f7ec-43c1-932d-bacdf69551e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:55.067010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48364a-f7ec-43c1-932d-bacdf69551e2 None admin] RBAC: Adding query filter params (role.id=d044315b993e4258ab44b9e5c0009df6, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=9d8bf513dc544632a09a2d10988a888f) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:55.067545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48364a-f7ec-43c1-932d-bacdf69551e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:55.070236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48364a-f7ec-43c1-932d-bacdf69551e2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:55.075787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:55.088438 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 520/1045] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:54 2022] GET /identity/v3/role_assignments?user.id=9d8bf513dc544632a09a2d10988a888f&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=d044315b993e4258ab44b9e5c0009df6 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:55.383194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc7124-52e5-4c27-af3c-c1cc1abc6d22 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:55.383580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc7124-52e5-4c27-af3c-c1cc1abc6d22 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:55.383750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc7124-52e5-4c27-af3c-c1cc1abc6d22 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:55.388433 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 526/1046] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:55.392241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae71f4-f59d-4fd9-b717-e45d3130bb1a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:55.392670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae71f4-f59d-4fd9-b717-e45d3130bb1a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:55.392851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae71f4-f59d-4fd9-b717-e45d3130bb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:55.413699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-23ae71f4-f59d-4fd9-b717-e45d3130bb1a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:55.414932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-23ae71f4-f59d-4fd9-b717-e45d3130bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:55.509425 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 521/1047] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:55.784999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc7124-52e5-4c27-af3c-c1cc1abc6d22 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:55.828322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0956e6-65ef-4a11-af8d-66054de7ed07 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:55.828865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0956e6-65ef-4a11-af8d-66054de7ed07 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:55.829093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0956e6-65ef-4a11-af8d-66054de7ed07 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:55.831129 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 522/1048] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:55.836698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca2609-e72b-4759-90c5-79fa318ec3a1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:55.836976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca2609-e72b-4759-90c5-79fa318ec3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:55.837249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca2609-e72b-4759-90c5-79fa318ec3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:55.856971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-abca2609-e72b-4759-90c5-79fa318ec3a1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:55.858185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-abca2609-e72b-4759-90c5-79fa318ec3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:55.900796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc7124-52e5-4c27-af3c-c1cc1abc6d22 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:55.932292 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 523/1049] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:55.941598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:55.942238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:55.942510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:55.942777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:55.953668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:55.958435 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:55.959404 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:55.961317 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 527/1050] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:55.966148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca2609-e72b-4759-90c5-79fa318ec3a1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.028072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.070558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca2609-e72b-4759-90c5-79fa318ec3a1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.108895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.109512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.109792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.110063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.110722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:56.111023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.112442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.121775 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 524/1051] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:56.127435 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.185319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.200140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.237098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ce85c-44dc-4247-a889-cc2f4175af58 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.274106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.274742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.275083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.275365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] PATH_INFO: `/v3/users/251d741a3b9c477ea4770278baaaeeef` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.289092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] RBAC: Authorizing `identity:get_user(user_id=251d741a3b9c477ea4770278baaaeeef)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.291099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.292822 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 528/1052] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:46:56 2022] GET /identity/v3/users/251d741a3b9c477ea4770278baaaeeef => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:56.298660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.309045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2b7fd-a7db-4832-b6bd-7ee585d86c3a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.334865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.349401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.350049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.350328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.350604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.351188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.353791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:56.362650 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:56.364621 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:56.365584 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 525/1053] 10.222.0.26 () {56 vars in 1105 bytes} [Sun Sep 25 00:46:56 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:56.369599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.416147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59a3f8-0e10-4b9e-af78-75c77ba9a380 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.427015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.451478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.452085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.452362 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.452661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.490203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9505c33-605b-4947-b350-37bd579650f8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.524211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.525934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.529904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.530915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.531401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.531884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.532775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:56.533305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.535574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.543211 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 526/1054] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:56.548967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.613630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.632450 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 529/1055] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 25 00:46:56.636209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.647669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.673537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee320c8a-a5f9-4d7e-92ef-73796a28d2b1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.708848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.709465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.709747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.710022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.710647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] RBAC: Adding query filter params (type=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:56.710950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.712315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.718121 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 527/1056] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:56.723642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.749270 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d025d-26f3-4a2b-8029-6c5a7dcdda06 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.791223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.792096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.792404 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.792688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.794846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] RBAC: Authorizing `identity:get_project(project_id=aecf09fd40234c5caf9dc6bb9495889d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.796592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.804238 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 530/1057] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:46:56 2022] GET /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:56.805598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:56.829100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb51c81-cf75-4718-989a-696dd5dfac9e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.846697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.871104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f9dce-7351-4787-8ef4-672135c84e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.871628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f9dce-7351-4787-8ef4-672135c84e5b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.871907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f9dce-7351-4787-8ef4-672135c84e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.872178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f9dce-7351-4787-8ef4-672135c84e5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.872882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f9dce-7351-4787-8ef4-672135c84e5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.874324 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f9dce-7351-4787-8ef4-672135c84e5b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.894005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:56.898204 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 528/1058] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:46:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:46:56.932242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a22b6a-4170-4c36-9aef-9f86851aaad2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:56.980188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958c387-1828-4dc5-a870-cb517905793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:56.981300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958c387-1828-4dc5-a870-cb517905793a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:56.981732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958c387-1828-4dc5-a870-cb517905793a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:56.982154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958c387-1828-4dc5-a870-cb517905793a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:56.983080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958c387-1828-4dc5-a870-cb517905793a None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=aecf09fd40234c5caf9dc6bb9495889d, user.id=251d741a3b9c477ea4770278baaaeeef) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:56.983555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958c387-1828-4dc5-a870-cb517905793a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:56.985203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958c387-1828-4dc5-a870-cb517905793a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:56.992000 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 531/1059] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:46:56 2022] GET /identity/v3/role_assignments?user.id=251d741a3b9c477ea4770278baaaeeef&scope.project.id=aecf09fd40234c5caf9dc6bb9495889d&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:46:58.020633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4586-c633-455a-84a4-050099ea0111 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:58.021163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4586-c633-455a-84a4-050099ea0111 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:58.021163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4586-c633-455a-84a4-050099ea0111 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:58.022659 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 529/1060] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:58.028152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb1a2-d285-49d8-a2c1-7676661b6fb5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:58.028599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb1a2-d285-49d8-a2c1-7676661b6fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:58.028796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0eb1a2-d285-49d8-a2c1-7676661b6fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:58.047448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ef0eb1a2-d285-49d8-a2c1-7676661b6fb5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:58.048914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ef0eb1a2-d285-49d8-a2c1-7676661b6fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:58.132430 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 532/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:46:58.401677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c4586-c633-455a-84a4-050099ea0111 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:58.522548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c4586-c633-455a-84a4-050099ea0111 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:58.561292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:58.561857 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:58.562137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:58.562408 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:58.574008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:58.578885 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:58.580065 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:46:58.581494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:58.582562 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 530/1062] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Sep 25 00:46:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:58.586125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0eb1a2-d285-49d8-a2c1-7676661b6fb5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:58.668082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:58.709629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0eb1a2-d285-49d8-a2c1-7676661b6fb5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:58.750760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:58.751387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:58.752256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:58.752256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:58.752641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:58.752948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:58.754365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:58.763945 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 533/1063] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Sep 25 00:46:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:58.770494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:58.830011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:58.941216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f179d26-1233-4276-aa68-ee4d15f75dec None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:58.990414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6febb-311a-4c42-a9e6-d9a5f531f876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:58.991063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6febb-311a-4c42-a9e6-d9a5f531f876 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:58.991345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6febb-311a-4c42-a9e6-d9a5f531f876 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:58.991618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6febb-311a-4c42-a9e6-d9a5f531f876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.005549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6febb-311a-4c42-a9e6-d9a5f531f876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:59.007002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6febb-311a-4c42-a9e6-d9a5f531f876 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6dc6febb-311a-4c42-a9e6-d9a5f531f876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:46:59.023220 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:59.025948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:46:59.026880 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:46:59.026880 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:46:59.026880 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:59.026880 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 531/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:59.033364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:59.163379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b96ff-674e-482b-970e-fbb0ffc1c6ee None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.163680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b96ff-674e-482b-970e-fbb0ffc1c6ee None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.164031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b96ff-674e-482b-970e-fbb0ffc1c6ee None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.165914 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 532/1065] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:46:59.168457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12129ece-6a3a-45b7-94ea-cb39871213b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:59.171685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd5c9-61b5-4b90-9a00-1cd4de351d63 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.171972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd5c9-61b5-4b90-9a00-1cd4de351d63 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.172255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd5c9-61b5-4b90-9a00-1cd4de351d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.191351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-564fd5c9-61b5-4b90-9a00-1cd4de351d63 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:59.193093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-564fd5c9-61b5-4b90-9a00-1cd4de351d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:59.208026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:59.215833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:59.216861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.217344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.217908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.218789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:59.219304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:59.220890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:59.234939 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 534/1066] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:59.240762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:59.279716 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 533/1067] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:59.281156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebc9c9-0371-4afb-b230-5052042befe5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.281524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebc9c9-0371-4afb-b230-5052042befe5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.282058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebc9c9-0371-4afb-b230-5052042befe5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.283297 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 534/1068] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:46:59.290848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c81e35-4559-422e-a419-f6e5eebf1148 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.291359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c81e35-4559-422e-a419-f6e5eebf1148 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.291791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c81e35-4559-422e-a419-f6e5eebf1148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.312226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e2c81e35-4559-422e-a419-f6e5eebf1148 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:46:59.313270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e2c81e35-4559-422e-a419-f6e5eebf1148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:46:59.322703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:59.402603 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 535/1069] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:46:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:46:59.407329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f80c-851d-4dac-95c7-c97aedfbdcc5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:59.447531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:59.448097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.448422 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.448694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.457266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:59.458769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:46:59.472572 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:59.473830 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:46:59.475453 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 535/1070] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:46:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:46:59.481925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c81e35-4559-422e-a419-f6e5eebf1148 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:59.579799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:59.625083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c81e35-4559-422e-a419-f6e5eebf1148 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:59.652277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:59.679327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:59.679981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.680261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.680669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.681418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:46:59.681723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:59.683177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:59.700961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:59.704908 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 536/1071] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:46:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:46:59.705823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:59.721936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe3df9-929e-404b-a7c2-26a99202008f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:59.763106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:59.763821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.765611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.765611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.843306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:46:59.844832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:46:59.848274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3019a3-278e-4674-9b5a-06abb7a4c16f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:46:59.901962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:46:59.902768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:46:59.903094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:46:59.903430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:46:59.916417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:46:59.917261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:46:59.921502 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:46:59.922666 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 537/1072] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:46:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:46:59.924022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:46:59.990795 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 536/1073] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:46:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 25 00:46:59.993678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.002584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:00.059993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b5998-c0fa-4d3e-af75-568fb1f073c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.096128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.096884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.097214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.097542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.106894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91db09-bc41-4baf-9d5f-65daff76f49f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.109071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.110555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:47:00.126494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:00.127719 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:47:00.129339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:47:00.129339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:00.129339 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:00.129339 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 538/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:00.129339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.143605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.144250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.144562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.144833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.145494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:00.145795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.147164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.156096 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 537/1075] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:46:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:00.156975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.174031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:00.242771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09a62-0325-4a6e-b8fa-5722648913b6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.257211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:00.279668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7effab-ab9a-44d8-b5b2-47b3c3259835 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.282615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.283278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.283555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.283826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.294905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.296391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:47:00.313132 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:00.314140 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:47:00.320643 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:47:00.320643 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:00.320643 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:00.320643 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 539/1076] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:00.320643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.320953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.320953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.320953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.321155 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.321794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:00.322162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.323547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.335862 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 538/1077] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:00.336646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.411783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:00.420775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ac06b-4722-41b7-a81d-6c2460445184 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.443026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8f8f7-f280-4f77-a31c-c675728cb19c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.455793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.456456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.456738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.457015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] PATH_INFO: `/v3/users/251d741a3b9c477ea4770278baaaeeef` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.458167 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] RBAC: Authorizing `identity:get_user(user_id=251d741a3b9c477ea4770278baaaeeef)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.459594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.461231 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 540/1078] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:00 2022] GET /identity/v3/users/251d741a3b9c477ea4770278baaaeeef => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:00.461231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.481103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.481719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.482004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.482284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.482930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:00.483232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.484647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.495205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:00.497311 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 539/1079] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:00.497975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.597307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a28ec-cfa3-4ac0-82df-f5550d3b3d56 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.635658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a2ae2-ffee-45b9-8f73-8e7b1d91e7a8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.638373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.638373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.638660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.639843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.650061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.652370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:47:00.665518 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:00.667312 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:47:00.668288 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:00.672509 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 541/1080] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:47:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:00.673422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.682497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:00.690172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.690940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.691295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.691632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.692912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] RBAC: Authorizing `identity:get_project(project_id=aecf09fd40234c5caf9dc6bb9495889d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.694385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.696346 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 540/1081] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:00 2022] GET /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d => generated 311 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:00.705439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.792293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:00.805984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bdee3-f60e-4b38-9e26-cf78eaf0b421 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.835071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b90bc3-2f91-4392-ace9-47c94c9422da None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:00.851448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.852067 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.852350 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.852677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.853900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:00.854203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.855642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.869556 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 542/1082] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:00.871308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9141213-3320-4e19-be55-dd11eb8865a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:00.871984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9141213-3320-4e19-be55-dd11eb8865a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:00.872267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9141213-3320-4e19-be55-dd11eb8865a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:00.872580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9141213-3320-4e19-be55-dd11eb8865a8 None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d/users/251d741a3b9c477ea4770278baaaeeef/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:00.874541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9141213-3320-4e19-be55-dd11eb8865a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=aecf09fd40234c5caf9dc6bb9495889d, user_id=251d741a3b9c477ea4770278baaaeeef, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:00.875630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:00.876035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9141213-3320-4e19-be55-dd11eb8865a8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:00.896025 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 541/1083] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:47:00 2022] PUT /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d/users/251d741a3b9c477ea4770278baaaeeef/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:47:00.996862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51375-0b29-4b56-80b5-60d271ecb64b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:01.035019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cb94e-03f1-4539-9390-2be4a110475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:01.035717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cb94e-03f1-4539-9390-2be4a110475c None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:01.036000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cb94e-03f1-4539-9390-2be4a110475c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:01.036274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cb94e-03f1-4539-9390-2be4a110475c None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/9d8bf513dc544632a09a2d10988a888f/roles/d044315b993e4258ab44b9e5c0009df6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:01.038621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cb94e-03f1-4539-9390-2be4a110475c None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=9d8bf513dc544632a09a2d10988a888f, role_id=d044315b993e4258ab44b9e5c0009df6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:01.040092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cb94e-03f1-4539-9390-2be4a110475c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:01.056023 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 543/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:47:00 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/9d8bf513dc544632a09a2d10988a888f/roles/d044315b993e4258ab44b9e5c0009df6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:47:03.102673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9494ab-bab8-46b7-a9b0-b345515b408c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:03.103903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9494ab-bab8-46b7-a9b0-b345515b408c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:03.104063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9494ab-bab8-46b7-a9b0-b345515b408c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:03.105474 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 542/1085] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:03.111408 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d195c-ce30-4f4c-ab49-e88552305554 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:03.111983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d195c-ce30-4f4c-ab49-e88552305554 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:03.111983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d195c-ce30-4f4c-ab49-e88552305554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:03.131589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-139d195c-ce30-4f4c-ab49-e88552305554 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:03.132841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-139d195c-ce30-4f4c-ab49-e88552305554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:03.142484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:03.227899 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 544/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:03.284112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615090b-4945-4cd1-bb5f-7c7170d66cd1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:03.284415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615090b-4945-4cd1-bb5f-7c7170d66cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:03.284697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615090b-4945-4cd1-bb5f-7c7170d66cd1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:03.286063 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 543/1087] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:03.291902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921abe8b-500f-4844-9ebc-ee93e7aebe3b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:03.292190 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921abe8b-500f-4844-9ebc-ee93e7aebe3b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:03.292509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921abe8b-500f-4844-9ebc-ee93e7aebe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:03.322828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-921abe8b-500f-4844-9ebc-ee93e7aebe3b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:03.324947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-921abe8b-500f-4844-9ebc-ee93e7aebe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:03.410908 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 545/1088] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:03.520544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615090b-4945-4cd1-bb5f-7c7170d66cd1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:03.591460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:03.638222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615090b-4945-4cd1-bb5f-7c7170d66cd1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:03.674460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:03.675017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:03.675295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:03.675562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:03.687570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:03.691774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921abe8b-500f-4844-9ebc-ee93e7aebe3b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:03.694601 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:03.695696 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:47:03.698242 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 544/1089] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:47:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:03.704510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:03.774195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:03.810011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921abe8b-500f-4844-9ebc-ee93e7aebe3b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:03.820742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589c1a7-7572-4275-88c3-43c1c80d556f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:03.843026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:03.857493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:03.858506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:03.858913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:03.859369 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:03.867975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:03.869176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:03.869632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:03.870080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:03.871234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:03.871716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:03.874936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:03.877319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:03.886088 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:03.887556 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:03.889658 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 546/1090] 10.222.0.26 () {56 vars in 1120 bytes} [Sun Sep 25 00:47:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:03.894480 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 545/1091] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:03.896621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:03.900919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.023962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5860665-6864-4a33-9d2e-09f211dc0708 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.041899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7394cd-f5ae-4386-baf1-92f565416e96 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.066928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.068005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.068255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.068768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] PATH_INFO: `/v3/users/251d741a3b9c477ea4770278baaaeeef` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.070435 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=251d741a3b9c477ea4770278baaaeeef)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.072840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.076436 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 546/1092] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:03 2022] GET /identity/v3/users/251d741a3b9c477ea4770278baaaeeef => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:04.083898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.087411 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:04.094195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.094817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.095098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.095407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.096135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:04.096484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.097915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.107233 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 547/1093] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Sep 25 00:47:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:04.113376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.146409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:04.221515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a9ddf-fd5d-478d-bee4-1544cfddf2a0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.251064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e753-fc2a-4f4c-9ed4-132b37cd4e23 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.279134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.280064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.280590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.281070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.295006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.295816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.296156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.296535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.314094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.316473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:47:04.340142 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:04.341252 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:47:04.342300 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:47:04.342300 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:04.342300 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:04.342300 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 548/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:47:04.349136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.387406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.389200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.427183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:04.474492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:04.507877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dee6-14d9-40ea-8877-06bc1afd1c53 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.551947 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 547/1095] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 25 00:47:04.558781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.566774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.567717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.567998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.568309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.569025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:04.569303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.570710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.592411 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 549/1096] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:04.600911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.706946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2964f33-d523-4e5a-8fc5-a49f4678e102 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.756539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.757232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.757560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.757773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.759255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] RBAC: Authorizing `identity:get_project(project_id=aecf09fd40234c5caf9dc6bb9495889d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.762108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.764117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f70939-6014-41c1-83e4-ecb2dbbac6e1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.774013 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 548/1097] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:04 2022] GET /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d => generated 311 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:04.774317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:04.776147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.806340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:04.808667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.809301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.809600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.809847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.820950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.822456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:47:04.835172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:04.836599 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:04.838364 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 550/1098] 10.222.0.26 () {56 vars in 1093 bytes} [Sun Sep 25 00:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:47:04.843405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:04.922513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419d1d5-8f1d-48f1-aace-43062af29b2e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:04.981947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:04.982561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:04.982848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:04.983125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:04.983905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=aecf09fd40234c5caf9dc6bb9495889d, user.id=251d741a3b9c477ea4770278baaaeeef) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:04.984236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:04.985925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:04.993407 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 549/1099] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:04 2022] GET /identity/v3/role_assignments?user.id=251d741a3b9c477ea4770278baaaeeef&scope.project.id=aecf09fd40234c5caf9dc6bb9495889d&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:04.996066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e5f58-c8b5-4544-af6f-c0cf1858cf58 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:05.040403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:05.042020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:05.042020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:05.042020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:05.043384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:05.043384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:05.044586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:05.070264 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 551/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Sun Sep 25 00:47:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:05.077075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:05.186944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:05.226181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf348bda-f4dc-4563-b91d-e521dd5dd1d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:05.275306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:05.276361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:05.276957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:05.277407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:05.380245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:05.381821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:05.482861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:05.515899 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 550/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:47:05.518723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:05.547372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:05.645196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea10e22-0502-4cc5-8366-1d3face5e031 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:05.691038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:05.694191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:05.694191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:05.694191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:05.708653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:05.712148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:47:05.731064 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:05.733689 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:47:05.735759 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:47:05.735759 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:47:05.735759 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:05.739135 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 552/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:05.741149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:05.879781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a59d-1575-41b6-9cbf-e9640fc589d4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:05.920647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:05.921182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:05.921458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:05.921746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:05.922444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:05.922874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:05.924192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:05.942974 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 551/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:05.945230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:06.044282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:06.080930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec4069-fdfa-4c73-abeb-ea0855707777 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:06.126861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:06.127894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:06.128445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:06.128965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:06.138995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:06.141317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:47:06.157521 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:06.165494 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:06.167260 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 553/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:06.169886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:06.225437 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:06.297732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3879d-87b6-4af2-be21-8f2c6e40531a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:06.347068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb498682-191e-4e9e-ad02-61aa0c13d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:06.347971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498682-191e-4e9e-ad02-61aa0c13d2fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:06.348423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498682-191e-4e9e-ad02-61aa0c13d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:06.348867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498682-191e-4e9e-ad02-61aa0c13d2fc None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:06.349618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb498682-191e-4e9e-ad02-61aa0c13d2fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:06.350090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb498682-191e-4e9e-ad02-61aa0c13d2fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:06.351831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb498682-191e-4e9e-ad02-61aa0c13d2fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:06.367891 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 552/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:06.374989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:06.502460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa7482-47ae-4db9-9e12-84e256e2b513 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:06.544371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb5e9-c654-42bc-b069-279c899c2f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:06.545303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb5e9-c654-42bc-b069-279c899c2f08 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:06.545303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb5e9-c654-42bc-b069-279c899c2f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:06.545610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb5e9-c654-42bc-b069-279c899c2f08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:06.546313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb5e9-c654-42bc-b069-279c899c2f08 None admin] RBAC: Adding query filter params (role.id=d044315b993e4258ab44b9e5c0009df6, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=9d8bf513dc544632a09a2d10988a888f) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:06.546620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb5e9-c654-42bc-b069-279c899c2f08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:06.548065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb5e9-c654-42bc-b069-279c899c2f08 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:06.551550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:06.556945 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 554/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:06 2022] GET /identity/v3/role_assignments?user.id=9d8bf513dc544632a09a2d10988a888f&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=d044315b993e4258ab44b9e5c0009df6 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:07.517291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180e31-5ae5-4952-a932-790de4a0ae57 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:07.517848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180e31-5ae5-4952-a932-790de4a0ae57 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:07.517848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180e31-5ae5-4952-a932-790de4a0ae57 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:07.519356 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 553/1107] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:07.526105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9bbc4-63eb-4858-a88e-6ff2cd1860d1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:07.526391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9bbc4-63eb-4858-a88e-6ff2cd1860d1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:07.526665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9bbc4-63eb-4858-a88e-6ff2cd1860d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:07.546978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eea9bbc4-63eb-4858-a88e-6ff2cd1860d1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:07.548209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eea9bbc4-63eb-4858-a88e-6ff2cd1860d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:07.631539 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 555/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:07.908702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df180e31-5ae5-4952-a932-790de4a0ae57 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:08.022460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df180e31-5ae5-4952-a932-790de4a0ae57 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:08.024112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:08.079060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bdfcd-ddf9-449f-b1e3-6e8eb1df21cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:08.080134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bdfcd-ddf9-449f-b1e3-6e8eb1df21cf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:08.080649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bdfcd-ddf9-449f-b1e3-6e8eb1df21cf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:08.081130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bdfcd-ddf9-449f-b1e3-6e8eb1df21cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:08.083063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115bdfcd-ddf9-449f-b1e3-6e8eb1df21cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:08.085743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115bdfcd-ddf9-449f-b1e3-6e8eb1df21cf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:08.089042 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 554/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:47:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:08.094781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9bbc4-63eb-4858-a88e-6ff2cd1860d1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:08.205445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9bbc4-63eb-4858-a88e-6ff2cd1860d1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:08.236396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:08.244049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d2a84-ca4c-4f6c-ab1c-a629e5c581a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:08.244716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d2a84-ca4c-4f6c-ab1c-a629e5c581a4 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:08.245002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d2a84-ca4c-4f6c-ab1c-a629e5c581a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:08.245279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d2a84-ca4c-4f6c-ab1c-a629e5c581a4 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:08.245971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d2a84-ca4c-4f6c-ab1c-a629e5c581a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:08.247470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d2a84-ca4c-4f6c-ab1c-a629e5c581a4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:08.272663 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 556/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:47:08 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:47:10.559586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778851-9dc7-4c3a-8785-68f29ded0fa3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:10.560124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778851-9dc7-4c3a-8785-68f29ded0fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:10.560124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778851-9dc7-4c3a-8785-68f29ded0fa3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:10.562074 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 555/1111] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:10.569706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc2843-1cc6-46fb-81f8-ad74e719d2aa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:10.570290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc2843-1cc6-46fb-81f8-ad74e719d2aa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:10.570290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc2843-1cc6-46fb-81f8-ad74e719d2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:10.594139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-25fc2843-1cc6-46fb-81f8-ad74e719d2aa None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:10.595403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-25fc2843-1cc6-46fb-81f8-ad74e719d2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:10.683022 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 557/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:10.973861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778851-9dc7-4c3a-8785-68f29ded0fa3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:11.100523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778851-9dc7-4c3a-8785-68f29ded0fa3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:11.140342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:11.141007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:11.141283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:11.141553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] PATH_INFO: `/v3/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:11.143343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] RBAC: Authorizing `identity:get_role(role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:11.144915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:11.146305 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 556/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:10 2022] GET /identity/v3/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 223 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:11.152705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc2843-1cc6-46fb-81f8-ad74e719d2aa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:11.273500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc2843-1cc6-46fb-81f8-ad74e719d2aa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:11.292179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:11.314541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:11.315230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:11.315542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:11.315849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] PATH_INFO: `/v3/users/10881ba4c72b43039efbec78d42015ec` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:11.331317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] RBAC: Authorizing `identity:get_user(user_id=10881ba4c72b43039efbec78d42015ec)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:11.332782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:11.334538 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 558/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:11 2022] GET /identity/v3/users/10881ba4c72b43039efbec78d42015ec => generated 286 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:11.342546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:11.393728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:11.465818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ace29-1148-46b7-8fe8-99a5991be15a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:11.504398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:11.505041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:11.505318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:11.505593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:11.589662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:11.591197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:11.638037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:11.712370 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 557/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:47:11.719416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:11.837120 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c94f-98bf-4ec7-87f2-7d0898ecb48e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:11.880961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6a165-32d1-415c-9c43-ab8273ade9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:11.881535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6a165-32d1-415c-9c43-ab8273ade9ba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:11.881881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6a165-32d1-415c-9c43-ab8273ade9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:11.882177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6a165-32d1-415c-9c43-ab8273ade9ba None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:11.883385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6a165-32d1-415c-9c43-ab8273ade9ba None admin] RBAC: Authorizing `identity:get_project(project_id=aecf09fd40234c5caf9dc6bb9495889d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:11.885434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6a165-32d1-415c-9c43-ab8273ade9ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:11.887174 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 559/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:11 2022] GET /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:11.894674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:12.016595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb3463-52ea-4c7b-bcf2-55b78d5a631f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:12.055139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e2588-53a9-41f9-82a4-bb75e8c0f47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:12.055767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e2588-53a9-41f9-82a4-bb75e8c0f47c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:12.056050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e2588-53a9-41f9-82a4-bb75e8c0f47c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:12.057033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e2588-53a9-41f9-82a4-bb75e8c0f47c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:12.057828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e2588-53a9-41f9-82a4-bb75e8c0f47c None admin] RBAC: Adding query filter params (role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=aecf09fd40234c5caf9dc6bb9495889d, user.id=10881ba4c72b43039efbec78d42015ec) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:12.058138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e2588-53a9-41f9-82a4-bb75e8c0f47c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:12.059596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e2588-53a9-41f9-82a4-bb75e8c0f47c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:12.066255 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 558/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:11 2022] GET /identity/v3/role_assignments?user.id=10881ba4c72b43039efbec78d42015ec&scope.project.id=aecf09fd40234c5caf9dc6bb9495889d&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:14.294412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38138f71-23fe-4849-81c7-cad2c6ae8f47 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:14.294990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38138f71-23fe-4849-81c7-cad2c6ae8f47 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:14.294990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38138f71-23fe-4849-81c7-cad2c6ae8f47 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:14.296285 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 560/1118] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:14.302006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb300a2-d969-44aa-9db9-23e19bfb6f75 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:14.302553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb300a2-d969-44aa-9db9-23e19bfb6f75 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:14.302553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb300a2-d969-44aa-9db9-23e19bfb6f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:14.309762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:14.322093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ccb300a2-d969-44aa-9db9-23e19bfb6f75 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:14.323305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ccb300a2-d969-44aa-9db9-23e19bfb6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:14.389950 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 559/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:47:14.666305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38138f71-23fe-4849-81c7-cad2c6ae8f47 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:14.712130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:14.775824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38138f71-23fe-4849-81c7-cad2c6ae8f47 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:14.813418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:14.814110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:14.814524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:14.814801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] PATH_INFO: `/v3/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:14.816553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] RBAC: Authorizing `identity:get_role(role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:14.818547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:14.820303 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 561/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:14 2022] GET /identity/v3/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:14.828519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb300a2-d969-44aa-9db9-23e19bfb6f75 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:14.957571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb300a2-d969-44aa-9db9-23e19bfb6f75 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:14.988216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:14.993574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:14.994770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:14.995045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:14.995312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] PATH_INFO: `/v3/users/10881ba4c72b43039efbec78d42015ec` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:14.996508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] RBAC: Authorizing `identity:get_user(user_id=10881ba4c72b43039efbec78d42015ec)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:14.997973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:14.999594 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 560/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:14 2022] GET /identity/v3/users/10881ba4c72b43039efbec78d42015ec => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:15.005493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:15.104867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83befdf0-3323-4fd0-b58e-b27b89b226d1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:15.116600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:15.140647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbcaa4-89a7-4f75-8800-abe1a4c67ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:15.141277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbcaa4-89a7-4f75-8800-abe1a4c67ffc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:15.141560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbcaa4-89a7-4f75-8800-abe1a4c67ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:15.141836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbcaa4-89a7-4f75-8800-abe1a4c67ffc None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:15.143233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbcaa4-89a7-4f75-8800-abe1a4c67ffc None admin] RBAC: Authorizing `identity:get_project(project_id=aecf09fd40234c5caf9dc6bb9495889d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:15.144692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbcaa4-89a7-4f75-8800-abe1a4c67ffc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:15.146401 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 562/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:15 2022] GET /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:15.152853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:15.252546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834220-f3c7-4bd1-bee8-7b558ff37b19 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:15.287030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69daf4b-0baa-4885-8ba9-b31706b463d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:15.287694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69daf4b-0baa-4885-8ba9-b31706b463d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:15.287972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69daf4b-0baa-4885-8ba9-b31706b463d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:15.288245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69daf4b-0baa-4885-8ba9-b31706b463d2 None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d/users/10881ba4c72b43039efbec78d42015ec/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:15.290414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69daf4b-0baa-4885-8ba9-b31706b463d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=aecf09fd40234c5caf9dc6bb9495889d, user_id=10881ba4c72b43039efbec78d42015ec, role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:15.292474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69daf4b-0baa-4885-8ba9-b31706b463d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:15.306074 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 561/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:47:15 2022] PUT /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d/users/10881ba4c72b43039efbec78d42015ec/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:47:17.503398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e07a5-6375-4a5d-91af-6df8822c9113 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:17.504108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e07a5-6375-4a5d-91af-6df8822c9113 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:17.504108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e07a5-6375-4a5d-91af-6df8822c9113 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:17.505279 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 563/1124] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:17.511023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619a77-1ca0-46ee-abdd-f3c87f700c64 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:17.511724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619a77-1ca0-46ee-abdd-f3c87f700c64 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:17.511724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf619a77-1ca0-46ee-abdd-f3c87f700c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:17.530631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bf619a77-1ca0-46ee-abdd-f3c87f700c64 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:17.531841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf619a77-1ca0-46ee-abdd-f3c87f700c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:17.590875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:17.602932 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 562/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:47:17.882631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e07a5-6375-4a5d-91af-6df8822c9113 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:17.995368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e07a5-6375-4a5d-91af-6df8822c9113 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:18.034197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:18.035039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:18.035442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:18.035757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] PATH_INFO: `/v3/roles/3a60d013f38c4183bf5b36d8e545a602` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:18.037222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] RBAC: Authorizing `identity:get_role(role_id=3a60d013f38c4183bf5b36d8e545a602)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:18.038593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:18.039943 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 564/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:17 2022] GET /identity/v3/roles/3a60d013f38c4183bf5b36d8e545a602 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:18.046368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf619a77-1ca0-46ee-abdd-f3c87f700c64 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:18.148988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf619a77-1ca0-46ee-abdd-f3c87f700c64 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:18.184536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:18.185229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:18.185531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:18.185812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] PATH_INFO: `/v3/users/10881ba4c72b43039efbec78d42015ec` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:18.187007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] RBAC: Authorizing `identity:get_user(user_id=10881ba4c72b43039efbec78d42015ec)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:18.188490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:18.190111 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 563/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:18 2022] GET /identity/v3/users/10881ba4c72b43039efbec78d42015ec => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:18.196035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:18.243621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:18.304233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a65fb-6b1e-45dd-80b5-82f7d1f1cdca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:18.346853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:18.347465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:18.347739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:18.348006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:18.421362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:18.422728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:18.469213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:18.528799 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 565/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:47:18.535089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:18.601909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:18.651946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f236cd3-5a14-4a7b-b594-52c1931dfb61 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:18.693733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b5fe1-9ece-4890-880f-f66350cbf7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:18.694290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b5fe1-9ece-4890-880f-f66350cbf7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:18.694574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b5fe1-9ece-4890-880f-f66350cbf7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:18.694852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b5fe1-9ece-4890-880f-f66350cbf7f6 None admin] PATH_INFO: `/v3/projects/aecf09fd40234c5caf9dc6bb9495889d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:18.695991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b5fe1-9ece-4890-880f-f66350cbf7f6 None admin] RBAC: Authorizing `identity:get_project(project_id=aecf09fd40234c5caf9dc6bb9495889d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:18.697514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b5fe1-9ece-4890-880f-f66350cbf7f6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:18.701981 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 564/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:18 2022] GET /identity/v3/projects/aecf09fd40234c5caf9dc6bb9495889d => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:18.708272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:18.809358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116115e-72ae-4421-a1e2-78939ce99643 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:18.847046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177d2d1-71fb-4bfe-b28e-f10005ea15eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:18.847675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d2d1-71fb-4bfe-b28e-f10005ea15eb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:18.847954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d2d1-71fb-4bfe-b28e-f10005ea15eb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:18.848226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d2d1-71fb-4bfe-b28e-f10005ea15eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:18.849009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177d2d1-71fb-4bfe-b28e-f10005ea15eb None admin] RBAC: Adding query filter params (role.id=3a60d013f38c4183bf5b36d8e545a602, scope.project.id=aecf09fd40234c5caf9dc6bb9495889d, user.id=10881ba4c72b43039efbec78d42015ec) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:18.849319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177d2d1-71fb-4bfe-b28e-f10005ea15eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:18.850795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177d2d1-71fb-4bfe-b28e-f10005ea15eb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:18.857569 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 566/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:18 2022] GET /identity/v3/role_assignments?user.id=10881ba4c72b43039efbec78d42015ec&scope.project.id=aecf09fd40234c5caf9dc6bb9495889d&role.id=3a60d013f38c4183bf5b36d8e545a602 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:21.060216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2767-c48f-4d44-babf-663aa9c96e91 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:21.060780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2767-c48f-4d44-babf-663aa9c96e91 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:21.060780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2767-c48f-4d44-babf-663aa9c96e91 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:21.062143 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 565/1131] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:21.067498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129aa50-0d9f-479e-8d47-07609334d07d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:21.067927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129aa50-0d9f-479e-8d47-07609334d07d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:21.068045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129aa50-0d9f-479e-8d47-07609334d07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:21.086214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7129aa50-0d9f-479e-8d47-07609334d07d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:21.087401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7129aa50-0d9f-479e-8d47-07609334d07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:21.090344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:21.158596 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 567/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:21.434126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b2767-c48f-4d44-babf-663aa9c96e91 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:21.539111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b2767-c48f-4d44-babf-663aa9c96e91 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:21.566603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:21.576260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a538c3-4dd3-4809-a51e-63c3ed2c5c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:21.576976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a538c3-4dd3-4809-a51e-63c3ed2c5c12 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:21.577264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a538c3-4dd3-4809-a51e-63c3ed2c5c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:21.577544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a538c3-4dd3-4809-a51e-63c3ed2c5c12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:21.578669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a538c3-4dd3-4809-a51e-63c3ed2c5c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:21.580112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a538c3-4dd3-4809-a51e-63c3ed2c5c12 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:21.581761 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 566/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:47:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:21.588455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129aa50-0d9f-479e-8d47-07609334d07d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:21.692443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129aa50-0d9f-479e-8d47-07609334d07d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:21.727483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1c81b-9376-4f0f-a6ae-fdc4df4a36b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:21.728097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c81b-9376-4f0f-a6ae-fdc4df4a36b7 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:21.728929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c81b-9376-4f0f-a6ae-fdc4df4a36b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:21.729196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c81b-9376-4f0f-a6ae-fdc4df4a36b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:21.729855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1c81b-9376-4f0f-a6ae-fdc4df4a36b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:21.731279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1c81b-9376-4f0f-a6ae-fdc4df4a36b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:21.743408 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:21.763945 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 568/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:47:21 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:47:24.016611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efe3a0-94e5-4e8f-a2a5-2125466fad84 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:24.017151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efe3a0-94e5-4e8f-a2a5-2125466fad84 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:24.017151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efe3a0-94e5-4e8f-a2a5-2125466fad84 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:24.018748 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 567/1135] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:24.024209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eaa7e-055a-48bf-8a34-00e531d68b1f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:24.024682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eaa7e-055a-48bf-8a34-00e531d68b1f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:24.024799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eaa7e-055a-48bf-8a34-00e531d68b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:24.043362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b69eaa7e-055a-48bf-8a34-00e531d68b1f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:24.044716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b69eaa7e-055a-48bf-8a34-00e531d68b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:24.113170 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 569/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:24.384063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efe3a0-94e5-4e8f-a2a5-2125466fad84 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:24.488987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efe3a0-94e5-4e8f-a2a5-2125466fad84 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:24.528986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513addf-ede4-46ac-a193-d612f78d0b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:24.530044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513addf-ede4-46ac-a193-d612f78d0b42 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:24.530524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513addf-ede4-46ac-a193-d612f78d0b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:24.531034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513addf-ede4-46ac-a193-d612f78d0b42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:24.533011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513addf-ede4-46ac-a193-d612f78d0b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:24.535442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513addf-ede4-46ac-a193-d612f78d0b42 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:24.539065 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 568/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:47:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:24.549662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eaa7e-055a-48bf-8a34-00e531d68b1f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:24.660548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eaa7e-055a-48bf-8a34-00e531d68b1f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:24.684592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:24.698116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661dd83-99bf-481d-b3d0-3978c6629e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:24.698734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661dd83-99bf-481d-b3d0-3978c6629e6f None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:24.699015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661dd83-99bf-481d-b3d0-3978c6629e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:24.699287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661dd83-99bf-481d-b3d0-3978c6629e6f None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:24.699959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661dd83-99bf-481d-b3d0-3978c6629e6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:24.701435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661dd83-99bf-481d-b3d0-3978c6629e6f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:24.724157 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 570/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:47:24 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:47:26.948336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbc4d8-6449-4074-b845-0e5afd049a7b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:26.948922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbc4d8-6449-4074-b845-0e5afd049a7b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:26.948922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbc4d8-6449-4074-b845-0e5afd049a7b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:26.950252 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 569/1139] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:26.955738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d438d4-d1c5-4ec4-82de-b56bb3a25b1c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:26.956127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d438d4-d1c5-4ec4-82de-b56bb3a25b1c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:26.956288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d438d4-d1c5-4ec4-82de-b56bb3a25b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:26.975428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f4d438d4-d1c5-4ec4-82de-b56bb3a25b1c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:26.976696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f4d438d4-d1c5-4ec4-82de-b56bb3a25b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:27.049319 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 571/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:27.324808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbc4d8-6449-4074-b845-0e5afd049a7b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:27.387408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:27.428050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbc4d8-6449-4074-b845-0e5afd049a7b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:27.464332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:27.464991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:27.465266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:27.465537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:27.475405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:27.480769 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:27.481877 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:47:27.482931 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 570/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:27.488091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d438d4-d1c5-4ec4-82de-b56bb3a25b1c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:27.593787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d438d4-d1c5-4ec4-82de-b56bb3a25b1c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:27.603237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:27.632245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:27.632907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:27.633186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:27.633455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:27.634134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:27.634434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:27.636412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:27.646089 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 572/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:27.653112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:27.751250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1ddb2-eea8-4806-8313-e0c9313b9a64 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:27.774713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:27.788038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:27.788713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:27.788990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:27.789261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] PATH_INFO: `/v3/users/8c73f752fc374c3eaae427a40790b2e7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:27.803073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=8c73f752fc374c3eaae427a40790b2e7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:27.804562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:27.806247 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 571/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:27 2022] GET /identity/v3/users/8c73f752fc374c3eaae427a40790b2e7 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:27.812197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:27.919556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e70cd-a1c3-44cf-b387-3207a70023b9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:27.952668 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:27.953279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:27.953562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:27.953839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:27.994570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:28.025668 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:28.027125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:28.128435 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 573/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:47:28.134491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:28.237196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1670b-d554-4106-8f07-d9c3b3fb1fc2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:28.272371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f1532-76c4-46b7-8bbb-a500c3f68982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:28.273027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1532-76c4-46b7-8bbb-a500c3f68982 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:28.273308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1532-76c4-46b7-8bbb-a500c3f68982 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:28.273577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1532-76c4-46b7-8bbb-a500c3f68982 None admin] PATH_INFO: `/v3/projects/91b2e972dff94a5180b816630cf078f3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:28.274720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f1532-76c4-46b7-8bbb-a500c3f68982 None admin] RBAC: Authorizing `identity:get_project(project_id=91b2e972dff94a5180b816630cf078f3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:28.276158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f1532-76c4-46b7-8bbb-a500c3f68982 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:28.277827 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 572/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:28 2022] GET /identity/v3/projects/91b2e972dff94a5180b816630cf078f3 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:28.284283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:28.364223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:28.415673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab09f1-eee2-472e-985f-17aac729a67e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:28.471585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed33bc-6a2b-4747-bea6-6d27b9eb125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:28.472617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed33bc-6a2b-4747-bea6-6d27b9eb125c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:28.472972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed33bc-6a2b-4747-bea6-6d27b9eb125c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:28.473239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed33bc-6a2b-4747-bea6-6d27b9eb125c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:28.473969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed33bc-6a2b-4747-bea6-6d27b9eb125c None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=91b2e972dff94a5180b816630cf078f3, user.id=8c73f752fc374c3eaae427a40790b2e7) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:28.474307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed33bc-6a2b-4747-bea6-6d27b9eb125c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:28.475712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed33bc-6a2b-4747-bea6-6d27b9eb125c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:28.482927 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 574/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:28 2022] GET /identity/v3/role_assignments?user.id=8c73f752fc374c3eaae427a40790b2e7&scope.project.id=91b2e972dff94a5180b816630cf078f3&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:30.667965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1900-9816-458a-8964-eb4311bd8655 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:30.668502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1900-9816-458a-8964-eb4311bd8655 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:30.668502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1900-9816-458a-8964-eb4311bd8655 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:30.669802 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 573/1147] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:30.675272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be364a-2686-47e0-a88d-d27884c2fcfb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:30.675704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be364a-2686-47e0-a88d-d27884c2fcfb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:30.675824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be364a-2686-47e0-a88d-d27884c2fcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:30.694499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b5be364a-2686-47e0-a88d-d27884c2fcfb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:30.695701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5be364a-2686-47e0-a88d-d27884c2fcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:30.763177 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 575/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:31.043932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1900-9816-458a-8964-eb4311bd8655 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:31.101704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:31.162492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1900-9816-458a-8964-eb4311bd8655 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:31.199759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:31.200424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:31.200707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:31.200978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:31.211820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:31.217134 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:31.218273 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:47:31.219162 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 574/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:31.224644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be364a-2686-47e0-a88d-d27884c2fcfb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:31.288118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:31.327503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be364a-2686-47e0-a88d-d27884c2fcfb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:31.362059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:31.362665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:31.362939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:31.363206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:31.363859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:31.364181 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:31.365573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:31.374375 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 576/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:31.380175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:31.484812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae9595-2818-46cd-9e0b-6adb24018e93 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:31.493658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:31.519623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:31.520252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:31.520553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:31.520826 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] PATH_INFO: `/v3/users/8c73f752fc374c3eaae427a40790b2e7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:31.521961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] RBAC: Authorizing `identity:get_user(user_id=8c73f752fc374c3eaae427a40790b2e7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:31.523397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:31.525020 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 575/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:31 2022] GET /identity/v3/users/8c73f752fc374c3eaae427a40790b2e7 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:31.531125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:31.654579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fb152-f32e-491a-b642-c24273109dc0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:31.683616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:31.693238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcc492-cddc-47a2-afc0-1471d99853fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:31.693878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcc492-cddc-47a2-afc0-1471d99853fe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:31.694162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcc492-cddc-47a2-afc0-1471d99853fe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:31.694438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcc492-cddc-47a2-afc0-1471d99853fe None admin] PATH_INFO: `/v3/projects/91b2e972dff94a5180b816630cf078f3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:31.695625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcc492-cddc-47a2-afc0-1471d99853fe None admin] RBAC: Authorizing `identity:get_project(project_id=91b2e972dff94a5180b816630cf078f3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:31.697151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcc492-cddc-47a2-afc0-1471d99853fe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:31.698858 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 577/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:31 2022] GET /identity/v3/projects/91b2e972dff94a5180b816630cf078f3 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:31.705298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:31.817178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60859856-0539-4eab-9ab9-79ea512ffb22 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:31.850684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6205-fd65-451b-8ec0-775bc8f36701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:31.851340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6205-fd65-451b-8ec0-775bc8f36701 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:31.851618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6205-fd65-451b-8ec0-775bc8f36701 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:31.851892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6205-fd65-451b-8ec0-775bc8f36701 None admin] PATH_INFO: `/v3/projects/91b2e972dff94a5180b816630cf078f3/users/8c73f752fc374c3eaae427a40790b2e7/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:31.853840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6205-fd65-451b-8ec0-775bc8f36701 None admin] RBAC: Authorizing `identity:create_grant(project_id=91b2e972dff94a5180b816630cf078f3, user_id=8c73f752fc374c3eaae427a40790b2e7, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:31.855277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6205-fd65-451b-8ec0-775bc8f36701 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:31.870275 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 576/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:47:31 2022] PUT /identity/v3/projects/91b2e972dff94a5180b816630cf078f3/users/8c73f752fc374c3eaae427a40790b2e7/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:47:34.090373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98ccb-7c59-46ce-83b5-a67208cf5f19 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:34.091025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98ccb-7c59-46ce-83b5-a67208cf5f19 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:34.091025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98ccb-7c59-46ce-83b5-a67208cf5f19 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:34.092242 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 578/1154] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:34.097717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d733b-e579-4690-8efc-f2bef463e1ea None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:34.098190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d733b-e579-4690-8efc-f2bef463e1ea None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:34.098321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d733b-e579-4690-8efc-f2bef463e1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:34.117206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-956d733b-e579-4690-8efc-f2bef463e1ea None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:34.118412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-956d733b-e579-4690-8efc-f2bef463e1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:34.146949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:34.187961 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 577/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:34.457019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd98ccb-7c59-46ce-83b5-a67208cf5f19 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:34.566038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd98ccb-7c59-46ce-83b5-a67208cf5f19 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:34.602645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:34.603304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:34.604130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:34.604427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:34.617844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:34.623374 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:34.624723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:34.626011 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 579/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:34.630921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d733b-e579-4690-8efc-f2bef463e1ea None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:34.750922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d733b-e579-4690-8efc-f2bef463e1ea None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:34.790619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:34.791231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:34.791511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:34.791784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:34.792476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:34.792783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:34.794169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:34.802327 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 578/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:34.809163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:34.867060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:34.914760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4d020-54d6-4bb7-986c-cf1895758af5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:34.951542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:34.952185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:34.952517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:34.952827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] PATH_INFO: `/v3/users/8c73f752fc374c3eaae427a40790b2e7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:34.954051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=8c73f752fc374c3eaae427a40790b2e7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:34.956242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:34.958726 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 580/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:34 2022] GET /identity/v3/users/8c73f752fc374c3eaae427a40790b2e7 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:34.965163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:34.994282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:35.066378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef090a-6475-4f50-8130-28a02753b486 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:35.100340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:35.101016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:35.101241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:35.101510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:35.166985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:35.168502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:35.194731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:35.265914 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 579/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:47:35.272509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:35.374295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf4a30-7e6c-4973-ae91-fd3604e03ec2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:35.399591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:35.412095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f30fc-dd44-4c77-bb24-742f8046033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:35.412793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f30fc-dd44-4c77-bb24-742f8046033a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:35.413073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f30fc-dd44-4c77-bb24-742f8046033a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:35.413345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f30fc-dd44-4c77-bb24-742f8046033a None admin] PATH_INFO: `/v3/projects/91b2e972dff94a5180b816630cf078f3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:35.414553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f30fc-dd44-4c77-bb24-742f8046033a None admin] RBAC: Authorizing `identity:get_project(project_id=91b2e972dff94a5180b816630cf078f3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:35.416566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f30fc-dd44-4c77-bb24-742f8046033a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:35.418284 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 581/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:35 2022] GET /identity/v3/projects/91b2e972dff94a5180b816630cf078f3 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:35.424957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:35.526270 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce834e-3743-490d-bf7f-a62924be1df4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:35.552891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:35.560792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b740c-ccaa-4939-bb44-e0c70746c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:35.561399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b740c-ccaa-4939-bb44-e0c70746c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:35.561680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b740c-ccaa-4939-bb44-e0c70746c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:35.561951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b740c-ccaa-4939-bb44-e0c70746c1c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:35.562675 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b740c-ccaa-4939-bb44-e0c70746c1c8 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=91b2e972dff94a5180b816630cf078f3, user.id=8c73f752fc374c3eaae427a40790b2e7) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:35.562980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b740c-ccaa-4939-bb44-e0c70746c1c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:35.564430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b740c-ccaa-4939-bb44-e0c70746c1c8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:35.570130 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 580/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:35 2022] GET /identity/v3/role_assignments?user.id=8c73f752fc374c3eaae427a40790b2e7&scope.project.id=91b2e972dff94a5180b816630cf078f3&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:37.785101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4cbf7-32dc-4642-8e4f-5ba9a1dc59d7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:37.785665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4cbf7-32dc-4642-8e4f-5ba9a1dc59d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:37.785665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4cbf7-32dc-4642-8e4f-5ba9a1dc59d7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:37.787004 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 582/1162] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:37.793868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3348f6e-7a01-4c7d-bf5d-9c1be91d3255 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:37.794573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3348f6e-7a01-4c7d-bf5d-9c1be91d3255 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:37.794573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3348f6e-7a01-4c7d-bf5d-9c1be91d3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:37.813638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d3348f6e-7a01-4c7d-bf5d-9c1be91d3255 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:37.814857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3348f6e-7a01-4c7d-bf5d-9c1be91d3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:37.894720 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 581/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:38.170276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4cbf7-32dc-4642-8e4f-5ba9a1dc59d7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:38.277739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4cbf7-32dc-4642-8e4f-5ba9a1dc59d7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:38.317562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:38.318174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:38.318472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:38.318723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:38.380698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:38.392355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:38.394098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:38.500051 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 583/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:47:38.506684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3348f6e-7a01-4c7d-bf5d-9c1be91d3255 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:38.616219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3348f6e-7a01-4c7d-bf5d-9c1be91d3255 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:38.640878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:38.656553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cdf25-227e-4084-9793-cb42a675868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:38.657192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cdf25-227e-4084-9793-cb42a675868f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:38.657472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cdf25-227e-4084-9793-cb42a675868f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:38.657746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cdf25-227e-4084-9793-cb42a675868f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:38.665065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cdf25-227e-4084-9793-cb42a675868f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:38.667164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cdf25-227e-4084-9793-cb42a675868f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9d9cdf25-227e-4084-9793-cb42a675868f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:38.679319 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:38.680592 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:47:38.682354 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 582/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:47:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:38.687189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:38.782737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:38.800524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa798b3-05c1-45f0-9eac-c0bd7af7cebf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:38.839723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696f589-fe1a-4f26-af6e-6ad941eb5fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:38.840338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f589-fe1a-4f26-af6e-6ad941eb5fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:38.840664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f589-fe1a-4f26-af6e-6ad941eb5fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:38.840934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f589-fe1a-4f26-af6e-6ad941eb5fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:38.841586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f589-fe1a-4f26-af6e-6ad941eb5fb4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:38.841883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f589-fe1a-4f26-af6e-6ad941eb5fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:38.843255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f589-fe1a-4f26-af6e-6ad941eb5fb4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:38.851011 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 584/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Sun Sep 25 00:47:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:41.054055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde72da-08e4-49a9-b188-5cbd69efe181 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:41.054624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde72da-08e4-49a9-b188-5cbd69efe181 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:41.054624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde72da-08e4-49a9-b188-5cbd69efe181 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:41.055935 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 583/1167] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:41.061722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945f203-0746-459f-823d-f9aad0f07487 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:41.062244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945f203-0746-459f-823d-f9aad0f07487 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:41.062244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945f203-0746-459f-823d-f9aad0f07487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:41.082095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f945f203-0746-459f-823d-f9aad0f07487 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:41.083294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f945f203-0746-459f-823d-f9aad0f07487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:41.152229 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 585/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:41.417688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde72da-08e4-49a9-b188-5cbd69efe181 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:41.521014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde72da-08e4-49a9-b188-5cbd69efe181 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:41.554817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a7f8-534d-4d62-8ca5-a6104dbcd85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:41.555432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a7f8-534d-4d62-8ca5-a6104dbcd85a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:41.555712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a7f8-534d-4d62-8ca5-a6104dbcd85a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:41.555992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a7f8-534d-4d62-8ca5-a6104dbcd85a None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:41.556757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72a7f8-534d-4d62-8ca5-a6104dbcd85a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:41.558095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72a7f8-534d-4d62-8ca5-a6104dbcd85a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:41.574223 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 584/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Sun Sep 25 00:47:41 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:47:43.772934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7051b-2351-4daa-a581-86cb1169747e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:43.773460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7051b-2351-4daa-a581-86cb1169747e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:43.773460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7051b-2351-4daa-a581-86cb1169747e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:43.774797 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 586/1170] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:43.780593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5547961-f019-42e7-8b6a-d6962732606f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:43.781079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5547961-f019-42e7-8b6a-d6962732606f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:43.781176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5547961-f019-42e7-8b6a-d6962732606f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:43.799506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d5547961-f019-42e7-8b6a-d6962732606f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:43.800736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d5547961-f019-42e7-8b6a-d6962732606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:43.837085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:43.875863 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 585/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:44.148166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7051b-2351-4daa-a581-86cb1169747e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:44.254696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:44.275346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7051b-2351-4daa-a581-86cb1169747e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:44.316340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fc541-0e19-4d2c-b657-13eeca4787c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:44.316997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc541-0e19-4d2c-b657-13eeca4787c0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:44.317274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc541-0e19-4d2c-b657-13eeca4787c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:44.317543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fc541-0e19-4d2c-b657-13eeca4787c0 None admin] PATH_INFO: `/v3/domains/baf8f3923a4446be909bd9aa8c85b7c8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:44.329693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fc541-0e19-4d2c-b657-13eeca4787c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=baf8f3923a4446be909bd9aa8c85b7c8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:44.331138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fc541-0e19-4d2c-b657-13eeca4787c0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:44.332842 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 587/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Sun Sep 25 00:47:44 2022] GET /identity/v3/domains/baf8f3923a4446be909bd9aa8c85b7c8 => generated 267 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:44.339495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5547961-f019-42e7-8b6a-d6962732606f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:44.445852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5547961-f019-42e7-8b6a-d6962732606f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:44.482962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85552c-be77-435b-82e8-1233938e984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:44.483587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85552c-be77-435b-82e8-1233938e984f None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:44.483872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85552c-be77-435b-82e8-1233938e984f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:44.484149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85552c-be77-435b-82e8-1233938e984f None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:44.484846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85552c-be77-435b-82e8-1233938e984f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:44.486351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85552c-be77-435b-82e8-1233938e984f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:44.518590 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 586/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:47:44 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:47:46.782965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561163b-dba6-485c-8063-c1b1214c7dba None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:46.783509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561163b-dba6-485c-8063-c1b1214c7dba None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:46.783509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561163b-dba6-485c-8063-c1b1214c7dba None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:46.784855 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 588/1174] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:46.790278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5be21f-d006-4f9c-b354-b6fb426188af None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:46.790715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5be21f-d006-4f9c-b354-b6fb426188af None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:46.790833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5be21f-d006-4f9c-b354-b6fb426188af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:46.809964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4c5be21f-d006-4f9c-b354-b6fb426188af None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:46.811169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4c5be21f-d006-4f9c-b354-b6fb426188af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:46.829766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:46.891850 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 587/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:47.159805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561163b-dba6-485c-8063-c1b1214c7dba None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:47.262489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561163b-dba6-485c-8063-c1b1214c7dba None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:47.297733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba071063-31c9-4c9f-a282-4dda6c252c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:47.298373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba071063-31c9-4c9f-a282-4dda6c252c25 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:47.298660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba071063-31c9-4c9f-a282-4dda6c252c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:47.298936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba071063-31c9-4c9f-a282-4dda6c252c25 None admin] PATH_INFO: `/v3/domains/baf8f3923a4446be909bd9aa8c85b7c8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:47.300087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba071063-31c9-4c9f-a282-4dda6c252c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=baf8f3923a4446be909bd9aa8c85b7c8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:47.301566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba071063-31c9-4c9f-a282-4dda6c252c25 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:47.303171 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 589/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Sun Sep 25 00:47:47 2022] GET /identity/v3/domains/baf8f3923a4446be909bd9aa8c85b7c8 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:47.309409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5be21f-d006-4f9c-b354-b6fb426188af None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:47.419068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5be21f-d006-4f9c-b354-b6fb426188af None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:47.453119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457f2fc-4f39-4430-ad66-9611e2e82dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:47.453917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457f2fc-4f39-4430-ad66-9611e2e82dda None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:47.454205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457f2fc-4f39-4430-ad66-9611e2e82dda None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:47.454577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457f2fc-4f39-4430-ad66-9611e2e82dda None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:47.455539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457f2fc-4f39-4430-ad66-9611e2e82dda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:47.457602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457f2fc-4f39-4430-ad66-9611e2e82dda None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:47.484910 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 588/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:47:47 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:47:49.721564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7271854-106e-4a9a-adb3-026981a52ae7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:49.722073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7271854-106e-4a9a-adb3-026981a52ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:49.722073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7271854-106e-4a9a-adb3-026981a52ae7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:49.723350 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 590/1178] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:49.729806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7642072-c950-41a0-a97b-324b5e421c29 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:49.730249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7642072-c950-41a0-a97b-324b5e421c29 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:49.730340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7642072-c950-41a0-a97b-324b5e421c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:49.749293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a7642072-c950-41a0-a97b-324b5e421c29 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:49.750512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a7642072-c950-41a0-a97b-324b5e421c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:49.753464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:49.821257 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 589/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:50.094985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7271854-106e-4a9a-adb3-026981a52ae7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:50.111070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:50.200785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7271854-106e-4a9a-adb3-026981a52ae7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:50.233889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:50.234511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:50.234785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:50.235053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:50.245385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:50.250100 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:50.251208 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:50.252124 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 591/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:47:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:50.257361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7642072-c950-41a0-a97b-324b5e421c29 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:50.356008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7642072-c950-41a0-a97b-324b5e421c29 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:50.389305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:50.389923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:50.390203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:50.390475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:50.391158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:50.391461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:50.392927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:50.403724 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 590/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:50.410051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:50.480907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:50.511221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e6e8c-0e56-4b2e-991b-f4b7182f9e2f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:50.544807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:50.545433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:50.545714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:50.545987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] PATH_INFO: `/v3/users/c104bb6a32f642f5b2dd30a837fd1001` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:50.558920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=c104bb6a32f642f5b2dd30a837fd1001)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:50.560351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:50.562003 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 592/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:50 2022] GET /identity/v3/users/c104bb6a32f642f5b2dd30a837fd1001 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:50.567878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:50.570781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:50.676957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c0f6f-d060-4bd2-ba82-b56fc4b7b34e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:50.709930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:50.710535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:50.710816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:50.711096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:50.793904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:50.795396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:50.797432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:50.910102 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 591/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:47:50.916266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:51.033115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c57843-5a49-4e8a-ae69-be7bb42a2fe2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:51.066071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:51.072897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee5be-846b-4325-831c-d1c91126dabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:51.073531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee5be-846b-4325-831c-d1c91126dabf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:51.073803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee5be-846b-4325-831c-d1c91126dabf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:51.074074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee5be-846b-4325-831c-d1c91126dabf None admin] PATH_INFO: `/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:51.075200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee5be-846b-4325-831c-d1c91126dabf None admin] RBAC: Authorizing `identity:get_project(project_id=9d2a4ae0bd9a46428c1c01dd18d202b2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:51.076663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee5be-846b-4325-831c-d1c91126dabf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:51.078293 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 593/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:50 2022] GET /identity/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:51.085089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:51.194965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2306ff-7b20-49c7-b6f0-36fdfc711cec None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:51.196570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:51.235760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcd26d-7429-4cfc-a237-e8c5d69698a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:51.236412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd26d-7429-4cfc-a237-e8c5d69698a0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:51.236700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd26d-7429-4cfc-a237-e8c5d69698a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:51.236972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd26d-7429-4cfc-a237-e8c5d69698a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:51.237714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcd26d-7429-4cfc-a237-e8c5d69698a0 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=9d2a4ae0bd9a46428c1c01dd18d202b2, user.id=c104bb6a32f642f5b2dd30a837fd1001) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:51.238019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcd26d-7429-4cfc-a237-e8c5d69698a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:51.239481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcd26d-7429-4cfc-a237-e8c5d69698a0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:51.246314 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 592/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:51 2022] GET /identity/v3/role_assignments?user.id=c104bb6a32f642f5b2dd30a837fd1001&scope.project.id=9d2a4ae0bd9a46428c1c01dd18d202b2&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:53.747430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb824f-dbf5-4b85-9010-b6ee087edf35 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:53.748213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb824f-dbf5-4b85-9010-b6ee087edf35 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:53.748213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb824f-dbf5-4b85-9010-b6ee087edf35 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:53.749407 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 594/1186] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:47:53.756622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27a42a-ff88-4ebf-8578-e62ff521139c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:53.757563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27a42a-ff88-4ebf-8578-e62ff521139c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:53.757563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27a42a-ff88-4ebf-8578-e62ff521139c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:53.790095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2f27a42a-ff88-4ebf-8578-e62ff521139c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:53.795169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2f27a42a-ff88-4ebf-8578-e62ff521139c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:53.933610 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 593/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:47:54.216829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb824f-dbf5-4b85-9010-b6ee087edf35 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:54.376261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb824f-dbf5-4b85-9010-b6ee087edf35 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:54.443579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:54.444597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:54.445066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:54.445522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:54.465583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:54.482109 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:54.484266 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:47:54.485735 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 595/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:47:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:54.494844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27a42a-ff88-4ebf-8578-e62ff521139c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:54.662549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:54.674079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27a42a-ff88-4ebf-8578-e62ff521139c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:54.733716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:54.734769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:54.735214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:54.735694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:54.738045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:54.738045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:54.739979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:54.755482 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 594/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:47:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:47:54.767885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:54.867562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:54.955213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e5b94-515b-4f6d-aab2-31a1fda4f2f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:55.007378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:55.007914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:55.008197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:55.008629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] PATH_INFO: `/v3/users/c104bb6a32f642f5b2dd30a837fd1001` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:55.009840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] RBAC: Authorizing `identity:get_user(user_id=c104bb6a32f642f5b2dd30a837fd1001)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:55.011511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:55.013455 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 596/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:54 2022] GET /identity/v3/users/c104bb6a32f642f5b2dd30a837fd1001 => generated 311 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:55.022943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:55.161608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe79d0-e9cd-47e1-91f6-a6fce1c5531b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:55.226701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1850a-5898-4c2b-90ce-6d89d1053e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:55.227421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1850a-5898-4c2b-90ce-6d89d1053e0d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:55.227705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1850a-5898-4c2b-90ce-6d89d1053e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:55.227985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1850a-5898-4c2b-90ce-6d89d1053e0d None admin] PATH_INFO: `/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:55.229624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1850a-5898-4c2b-90ce-6d89d1053e0d None admin] RBAC: Authorizing `identity:get_project(project_id=9d2a4ae0bd9a46428c1c01dd18d202b2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:55.232293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1850a-5898-4c2b-90ce-6d89d1053e0d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:55.235567 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 595/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:55 2022] GET /identity/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2 => generated 361 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:55.246542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:55.408535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b66c2-8519-43cb-b752-b5b02cad9eab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:55.457326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:55.473333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9928ad-d667-4101-b2d5-081ce7420431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:55.474372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9928ad-d667-4101-b2d5-081ce7420431 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:55.474811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9928ad-d667-4101-b2d5-081ce7420431 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:55.475244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9928ad-d667-4101-b2d5-081ce7420431 None admin] PATH_INFO: `/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2/users/c104bb6a32f642f5b2dd30a837fd1001/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:55.478623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9928ad-d667-4101-b2d5-081ce7420431 None admin] RBAC: Authorizing `identity:create_grant(project_id=9d2a4ae0bd9a46428c1c01dd18d202b2, user_id=c104bb6a32f642f5b2dd30a837fd1001, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:55.481225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9928ad-d667-4101-b2d5-081ce7420431 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:55.504801 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 597/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:47:55 2022] PUT /identity/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2/users/c104bb6a32f642f5b2dd30a837fd1001/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:47:57.853517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d81a-d824-4d6e-ad9c-de3b80b61226 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:57.854869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d81a-d824-4d6e-ad9c-de3b80b61226 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:57.854869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d81a-d824-4d6e-ad9c-de3b80b61226 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:57.856329 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 596/1193] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:47:57.863597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ab320-a07c-4f6f-a581-efc78e2c4e81 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:57.864184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ab320-a07c-4f6f-a581-efc78e2c4e81 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:57.864405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ab320-a07c-4f6f-a581-efc78e2c4e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:57.890263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7a8ab320-a07c-4f6f-a581-efc78e2c4e81 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:47:57.891583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a8ab320-a07c-4f6f-a581-efc78e2c4e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:47:57.978314 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 598/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:47:58.258033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d81a-d824-4d6e-ad9c-de3b80b61226 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:58.291360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:58.431006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d81a-d824-4d6e-ad9c-de3b80b61226 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:58.486972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:58.488038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:58.488580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:58.488971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:58.508980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:47:58.516835 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:47:58.518454 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:47:58.519939 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 597/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:47:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:47:58.527794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ab320-a07c-4f6f-a581-efc78e2c4e81 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:58.664946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ab320-a07c-4f6f-a581-efc78e2c4e81 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:58.698738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:58.723764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:58.724421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:58.724700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:58.724975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:58.725667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:47:58.725969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:58.727352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:58.738483 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 599/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:47:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:58.748740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:58.874199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:58.900490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d7dfd-62bb-41b9-93a6-551bab454689 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:58.961878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:58.962579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:58.962922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:58.964001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] PATH_INFO: `/v3/users/c104bb6a32f642f5b2dd30a837fd1001` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:58.965693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] RBAC: Authorizing `identity:get_user(user_id=c104bb6a32f642f5b2dd30a837fd1001)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:58.968216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:58.970884 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 598/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Sun Sep 25 00:47:58 2022] GET /identity/v3/users/c104bb6a32f642f5b2dd30a837fd1001 => generated 311 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:58.977859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:59.171475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7926191-52c7-4003-897d-0be7368e004e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:59.242049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:59.243074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:59.243607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:59.244184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:59.346908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:59.393605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:59.395910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:59.557327 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 600/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:47:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 25 00:47:59.566969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:59.707905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0165bfb-82c7-4c37-9608-ae1e4687904c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:47:59.756930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b9cc6-baf6-46b9-9eba-c67244fb32a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:47:59.758094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b9cc6-baf6-46b9-9eba-c67244fb32a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:47:59.758718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b9cc6-baf6-46b9-9eba-c67244fb32a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:47:59.759020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b9cc6-baf6-46b9-9eba-c67244fb32a7 None admin] PATH_INFO: `/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:47:59.761062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b9cc6-baf6-46b9-9eba-c67244fb32a7 None admin] RBAC: Authorizing `identity:get_project(project_id=9d2a4ae0bd9a46428c1c01dd18d202b2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:47:59.763747 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b9cc6-baf6-46b9-9eba-c67244fb32a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:47:59.767434 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 599/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Sun Sep 25 00:47:59 2022] GET /identity/v3/projects/9d2a4ae0bd9a46428c1c01dd18d202b2 => generated 361 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:47:59.779967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:47:59.912464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:47:59.957583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd020044-aa99-4d18-822e-b0515b3a9d84 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:48:00.029698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8a958-74bd-415e-bbd2-220ec770d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:48:00.030658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8a958-74bd-415e-bbd2-220ec770d539 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:48:00.031132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8a958-74bd-415e-bbd2-220ec770d539 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:48:00.031608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8a958-74bd-415e-bbd2-220ec770d539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:48:00.033007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8a958-74bd-415e-bbd2-220ec770d539 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=9d2a4ae0bd9a46428c1c01dd18d202b2, user.id=c104bb6a32f642f5b2dd30a837fd1001) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:48:00.033563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8a958-74bd-415e-bbd2-220ec770d539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:48:00.036318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8a958-74bd-415e-bbd2-220ec770d539 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:48:00.048182 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 601/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:47:59 2022] GET /identity/v3/role_assignments?user.id=c104bb6a32f642f5b2dd30a837fd1001&scope.project.id=9d2a4ae0bd9a46428c1c01dd18d202b2&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:48:09.749576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:10.042535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=29 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:05.188432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3750e71-81e0-460a-838f-34a0c9845994 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:05.188432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3750e71-81e0-460a-838f-34a0c9845994 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:05.188432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3750e71-81e0-460a-838f-34a0c9845994 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:05.189786 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 600/1201] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:05.203343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f1678-7622-4d9f-95e4-e517bae6a8e3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:05.204405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f1678-7622-4d9f-95e4-e517bae6a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:05.204718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f1678-7622-4d9f-95e4-e517bae6a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:05.248817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4d1f1678-7622-4d9f-95e4-e517bae6a8e3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:05.250572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4d1f1678-7622-4d9f-95e4-e517bae6a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:05.444923 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 602/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:05.854816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3750e71-81e0-460a-838f-34a0c9845994 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:06.066909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3750e71-81e0-460a-838f-34a0c9845994 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:06.128581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:06.129979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:06.130591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:06.131196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:06.153174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:06.162435 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:06.192554 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:06.194497 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:06.196049 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:06.196049 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:06.196049 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:06.196049 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 601/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:06.204473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f1678-7622-4d9f-95e4-e517bae6a8e3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:06.440918 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f1678-7622-4d9f-95e4-e517bae6a8e3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:06.494569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:06.507014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356730d6-483c-43f7-84c6-0e8b8fd1cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:06.507864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356730d6-483c-43f7-84c6-0e8b8fd1cb90 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:06.508535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356730d6-483c-43f7-84c6-0e8b8fd1cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:06.508994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356730d6-483c-43f7-84c6-0e8b8fd1cb90 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:06.510076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356730d6-483c-43f7-84c6-0e8b8fd1cb90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:06.511328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356730d6-483c-43f7-84c6-0e8b8fd1cb90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:06.513027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356730d6-483c-43f7-84c6-0e8b8fd1cb90 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:06.529865 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 603/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:06.537105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:06.613680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:06.697905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a2b9a-d4fb-4b2e-898d-cab6c6f4b75d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:06.762874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a53620-7439-481a-bb40-0e9a4eabcd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:06.764080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a53620-7439-481a-bb40-0e9a4eabcd37 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:06.764472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a53620-7439-481a-bb40-0e9a4eabcd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:06.765023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a53620-7439-481a-bb40-0e9a4eabcd37 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:06.766029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a53620-7439-481a-bb40-0e9a4eabcd37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:06.768223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a53620-7439-481a-bb40-0e9a4eabcd37 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:06.808865 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 602/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Sun Sep 25 00:49:06 2022] POST /identity/v3/users => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:49:09.648118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0854-dea6-4770-af31-91cd4a44a9a1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:09.648118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0854-dea6-4770-af31-91cd4a44a9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:09.648118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0854-dea6-4770-af31-91cd4a44a9a1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:09.648880 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 604/1206] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:49:09.655135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28330a73-9225-4d65-a28a-164bad961c19 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:09.659069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28330a73-9225-4d65-a28a-164bad961c19 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:09.659069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28330a73-9225-4d65-a28a-164bad961c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:09.677653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-28330a73-9225-4d65-a28a-164bad961c19 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:09.679551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-28330a73-9225-4d65-a28a-164bad961c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:09.796464 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 603/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:10.140900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0854-dea6-4770-af31-91cd4a44a9a1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:10.323946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0854-dea6-4770-af31-91cd4a44a9a1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:10.379097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:10.379859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:10.380320 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:10.380815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:10.398381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:49:10.404292 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:10.406438 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:10.408203 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 605/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:49:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:49:10.417918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28330a73-9225-4d65-a28a-164bad961c19 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:10.493691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:10.602763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28330a73-9225-4d65-a28a-164bad961c19 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:10.653730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:10.656763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:10.656763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:10.656763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:10.656763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:10.656763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:10.657960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:10.668242 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 604/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:49:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:10.674572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:10.740644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:10.818580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fef613-c9d4-43b2-97fd-c496fff8db9b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:10.879472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:10.880538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:10.881165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:10.881718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:10.906681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:10.908504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:10.938598 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:10.941886 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:10.944081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:10.944081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:10.944081 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:10.944081 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 606/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:10.947238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:11.086883 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:11.091772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edaa10-1780-466b-b5f4-2b28accf9b0c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:11.141077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:11.141708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:11.141994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:11.142266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:11.142961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:11.143263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:11.144699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:11.160530 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 605/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:11.169241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:11.351223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14158c5d-f0ae-4c25-b675-8037d0c85d14 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:11.353996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:11.412842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:11.414029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:11.414612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:11.415080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:11.430465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:11.433471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:49:11.454514 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:11.456334 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:11.457948 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 607/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:11.466571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:11.633351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02423e41-bf74-4e08-a1d3-4cb8183bf6e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:11.703245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:11.704760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:11.704760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:11.705014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:11.708878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:11.709670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:11.712614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:11.740774 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 606/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Sep 25 00:49:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:11.748573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:11.938418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c044585-33ed-480c-bb59-209329e77552 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:11.995743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:11.996228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:11.996552 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:11.996899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:12.043460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:12.094326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:12.095978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:12.303499 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 608/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:49:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:12.310898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:12.338096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:12.444646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb991b03-6933-4be8-8937-0602d3e84cbf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:12.493710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:12.494397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:12.494710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:12.495007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:12.512741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:12.515790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:12.548070 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:12.549526 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:12.550506 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:12.550506 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:12.550506 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:12.550506 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 607/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:12.558679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:12.669023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:12.744787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b4cab-1452-4390-83b7-cbcee1c967b0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:12.795204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:12.796256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:12.796774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:12.797230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:12.798448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:12.799119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:12.802516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:12.829385 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 609/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:12.840219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:12.973698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:13.024061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd21ac-16e8-4323-915a-86eba431539b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:13.085131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:13.085845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:13.086293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:13.086663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:13.095885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:13.098307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:49:13.112723 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:13.115409 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:13.117658 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 608/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:49:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:13.121127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:13.307289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24c4c-295a-45d0-a343-a9455ee6d53e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:13.340286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:13.358582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62318f2c-66df-41ea-beb3-6110205c000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:13.359215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318f2c-66df-41ea-beb3-6110205c000c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:13.359563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318f2c-66df-41ea-beb3-6110205c000c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:13.359841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318f2c-66df-41ea-beb3-6110205c000c None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:13.360637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318f2c-66df-41ea-beb3-6110205c000c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:13.360945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318f2c-66df-41ea-beb3-6110205c000c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:13.362756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318f2c-66df-41ea-beb3-6110205c000c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:13.378276 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 610/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:49:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:13.384915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:13.545073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc10e0-1add-4928-9795-751713da8031 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:13.586971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:13.600025 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba350889-cc0c-430d-8f53-c84c13ad05dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:13.600660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba350889-cc0c-430d-8f53-c84c13ad05dc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:13.601067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba350889-cc0c-430d-8f53-c84c13ad05dc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:13.601453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba350889-cc0c-430d-8f53-c84c13ad05dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:13.602402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba350889-cc0c-430d-8f53-c84c13ad05dc None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=d45df63e9f2449be8d13f58100775eb7) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:13.602839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba350889-cc0c-430d-8f53-c84c13ad05dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:13.605308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba350889-cc0c-430d-8f53-c84c13ad05dc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:13.613148 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 609/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:49:13 2022] GET /identity/v3/role_assignments?user.id=d45df63e9f2449be8d13f58100775eb7&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:16.101988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650b327-edff-46ad-874c-97b425678a47 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:16.102661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650b327-edff-46ad-874c-97b425678a47 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:16.102661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650b327-edff-46ad-874c-97b425678a47 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:16.108450 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 611/1220] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:16.111971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58bd5-6c7e-4162-b4bb-586d4d3187f3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:16.112503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58bd5-6c7e-4162-b4bb-586d4d3187f3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:16.112899 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58bd5-6c7e-4162-b4bb-586d4d3187f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:16.135571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7ea58bd5-6c7e-4162-b4bb-586d4d3187f3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:16.137186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7ea58bd5-6c7e-4162-b4bb-586d4d3187f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:16.230896 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 610/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:16.587745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650b327-edff-46ad-874c-97b425678a47 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:16.784049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650b327-edff-46ad-874c-97b425678a47 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:16.852322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:16.853068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:16.853506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:16.853974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:16.873073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:49:16.878527 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:16.880170 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:49:16.881625 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:16.884616 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 612/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:49:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:16.888507 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58bd5-6c7e-4162-b4bb-586d4d3187f3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:17.064637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58bd5-6c7e-4162-b4bb-586d4d3187f3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:17.103888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:17.128632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:17.129907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:17.129907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:17.129907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:17.131492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:17.131492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:17.133750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:17.149236 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 611/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:49:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:17.156287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:17.228815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:17.334690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82f62-863a-4b39-b544-b529ab5e090f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:17.399375 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:17.400070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:17.400367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:17.400709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:17.413126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:17.414614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:17.432586 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:17.434546 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:17.436172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:17.436172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:17.436172 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:17.436172 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 613/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:17.440058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:17.619033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508badb3-650b-4877-8cc8-896029a056fb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:17.686396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:17.686658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:17.687114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:17.687584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:17.688476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:17.689022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:17.691069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:17.711557 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 612/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:17.720587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:17.911591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:17.922006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae172c-39d1-4aee-9a5b-8c0b6a939159 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:17.978561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:17.979295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:17.980168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:17.980508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:17.995654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:17.997640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:49:18.011692 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:18.013872 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:18.015305 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 614/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:18.020977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:18.095496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:18.188299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8e915-5523-418b-bbe8-db37757363f4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:18.256449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:18.257373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:18.257801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:18.258083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:18.259017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:18.259325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:18.261114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:18.293436 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 613/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Sep 25 00:49:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:18.296091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:18.460986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd40956-3f1e-4851-8c99-a8580051995c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:18.512670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:18.513271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:18.513588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:18.513905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:18.528228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:18.529762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:18.545413 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:18.546943 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:18.548063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:18.548063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:18.548063 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:18.551420 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 615/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:18.552799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:18.692061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5e61a-d4a8-41c1-87d1-22cca11f5ca7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:18.694807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:18.746974 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:18.747602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:18.747888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:18.748163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:18.748904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:18.749241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:18.750661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:18.764244 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 614/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:18.770924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:18.786724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:18.967815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b125202-7f3e-4c2c-84da-36b276027b40 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:19.024955 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:19.025854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:19.026191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:19.026988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:19.037344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:19.038933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:49:19.055807 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:19.057529 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:19.059723 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 616/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:49:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:19.066600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:19.239518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f746e1-082e-4564-8a89-a5cee9984693 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:19.291648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad631f1-9e7f-4570-b0fd-c79ac99302f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:19.292949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad631f1-9e7f-4570-b0fd-c79ac99302f2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:19.292949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad631f1-9e7f-4570-b0fd-c79ac99302f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:19.293211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad631f1-9e7f-4570-b0fd-c79ac99302f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:19.294517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad631f1-9e7f-4570-b0fd-c79ac99302f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:19.294976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad631f1-9e7f-4570-b0fd-c79ac99302f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:19.296765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad631f1-9e7f-4570-b0fd-c79ac99302f2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:19.313486 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 615/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:49:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:19.320268 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:19.411545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:19.477003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fedd47-8591-4cfd-b319-75b3bd0f73f1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:19.533939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f0e55-3fb0-42b5-b69e-762777e6c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:19.534817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0e55-3fb0-42b5-b69e-762777e6c46d None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:19.535201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0e55-3fb0-42b5-b69e-762777e6c46d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:19.535509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0e55-3fb0-42b5-b69e-762777e6c46d None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/d45df63e9f2449be8d13f58100775eb7/roles/33e3a06092a146018396b00ae577d199` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:19.554942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f0e55-3fb0-42b5-b69e-762777e6c46d None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=d45df63e9f2449be8d13f58100775eb7, role_id=33e3a06092a146018396b00ae577d199)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:19.556722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f0e55-3fb0-42b5-b69e-762777e6c46d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:19.576454 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 617/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:49:19 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/d45df63e9f2449be8d13f58100775eb7/roles/33e3a06092a146018396b00ae577d199 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:49:22.452234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8192a-7103-497b-ad74-140a4851e503 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:22.452904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8192a-7103-497b-ad74-140a4851e503 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:22.452904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8192a-7103-497b-ad74-140a4851e503 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:22.454371 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 616/1233] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:22.461002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ef096-d99e-41e2-8c1c-30f06486a51a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:22.461564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ef096-d99e-41e2-8c1c-30f06486a51a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:22.461564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ef096-d99e-41e2-8c1c-30f06486a51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:22.484552 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-131ef096-d99e-41e2-8c1c-30f06486a51a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:22.485892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-131ef096-d99e-41e2-8c1c-30f06486a51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:22.605429 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 618/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:22.919540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8192a-7103-497b-ad74-140a4851e503 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:22.969309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:23.082737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8192a-7103-497b-ad74-140a4851e503 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:23.131007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:23.131786 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:23.132079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:23.132358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:23.149724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:49:23.154870 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:23.157077 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:23.158742 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 617/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Sun Sep 25 00:49:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:23.162621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ef096-d99e-41e2-8c1c-30f06486a51a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:23.261461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:23.358286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ef096-d99e-41e2-8c1c-30f06486a51a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:23.406733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:23.407382 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:23.407667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:23.407943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:23.408710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:23.409044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:23.410623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:23.421987 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 619/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Sun Sep 25 00:49:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:23.428914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:23.530188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:23.549805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adab97-288e-4bec-b2ea-b657caff108a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:23.595725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:23.596766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:23.597049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:23.597499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:23.614407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:23.616509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:23.643714 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:23.645361 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:23.646612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:23.646612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:23.646612 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:23.646612 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 618/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:23.653478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:23.825871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539b627-c988-4c14-8e95-801c54765a1e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:23.840420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:23.872967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:23.873634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:23.873932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:23.874265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:23.874834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:23.875174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:23.876503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:23.894168 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 620/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:23.904074 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:24.037540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141e951-cbe3-4e54-9353-29fbcadc995b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:24.079106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:24.079649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:24.079945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:24.080199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:24.083316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:24.089857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:24.091546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:49:24.106177 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:24.107418 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:24.108480 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 619/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:24.116096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:24.253777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0e94-5acd-4711-9672-4a3bd93f6650 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:24.315920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:24.316840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:24.317225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:24.317580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:24.318614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:24.319139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:24.321771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:24.358401 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 621/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Sep 25 00:49:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:24.367875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:24.560085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c631e9f-6364-4e9b-b065-1313ed57057b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:24.598198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:24.598864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:24.599119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:24.599355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:24.674228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:24.682582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:24.684162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:24.840948 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 620/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:49:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:24.848013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:24.886881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:25.010711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20673018-6ff1-493f-b322-2ebc5eed4efc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:25.068557 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:25.069210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:25.069493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:25.069767 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:25.081455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:25.082898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:25.097382 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:25.098672 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:25.099712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:25.099712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:25.099712 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:25.100537 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 622/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:25.105456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:25.222587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:25.229112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6515df-05dd-403f-a034-ef5818bb9b29 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:25.272198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:25.272908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:25.273461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:25.273722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:25.274361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:25.274657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:25.276058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:25.290936 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 621/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:25.298782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:25.451835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:25.485838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1d9b-d55f-487d-b93d-c989f22d75aa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:25.541159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:25.542293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:25.542679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:25.542959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:25.551309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:25.552822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:49:25.567823 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:25.569817 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:25.571764 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 623/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:49:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:25.576147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:25.717938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e47142-05d5-4ef5-ae84-758c491825e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:25.775992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceaaee1-63ee-4795-a22e-4b337a315d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:25.777307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaaee1-63ee-4795-a22e-4b337a315d17 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:25.777710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaaee1-63ee-4795-a22e-4b337a315d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:25.778111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaaee1-63ee-4795-a22e-4b337a315d17 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:25.778920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceaaee1-63ee-4795-a22e-4b337a315d17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:25.779339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceaaee1-63ee-4795-a22e-4b337a315d17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:25.780929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceaaee1-63ee-4795-a22e-4b337a315d17 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:25.797389 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 622/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:49:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:25.806456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:25.970133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14faa8e-2c07-4b88-b6a4-83ee7fcc7c98 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:26.032953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab206f0-9f09-4c8b-9311-37e88319e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:26.033676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab206f0-9f09-4c8b-9311-37e88319e6da None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:26.037593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab206f0-9f09-4c8b-9311-37e88319e6da None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:26.037593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab206f0-9f09-4c8b-9311-37e88319e6da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:26.037593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab206f0-9f09-4c8b-9311-37e88319e6da None admin] RBAC: Adding query filter params (role.id=33e3a06092a146018396b00ae577d199, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=d45df63e9f2449be8d13f58100775eb7) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:26.037593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab206f0-9f09-4c8b-9311-37e88319e6da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:26.041448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab206f0-9f09-4c8b-9311-37e88319e6da None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:26.051082 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 624/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:49:25 2022] GET /identity/v3/role_assignments?user.id=d45df63e9f2449be8d13f58100775eb7&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=33e3a06092a146018396b00ae577d199 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:28.883512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4a088-d1d9-4aaf-8b63-8e66a16d687d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:28.884421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4a088-d1d9-4aaf-8b63-8e66a16d687d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:28.884421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f4a088-d1d9-4aaf-8b63-8e66a16d687d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:28.885985 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 623/1247] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:28.893057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb1816-925f-4059-803c-4f16d86363ad None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:28.893834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb1816-925f-4059-803c-4f16d86363ad None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:28.893996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb1816-925f-4059-803c-4f16d86363ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:28.903577 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:28.923342 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-37bb1816-925f-4059-803c-4f16d86363ad None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:28.925048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-37bb1816-925f-4059-803c-4f16d86363ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:29.044855 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 625/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:29.388551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4a088-d1d9-4aaf-8b63-8e66a16d687d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:29.411005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:29.511826 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f4a088-d1d9-4aaf-8b63-8e66a16d687d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:29.556008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:29.556715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:29.557000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:29.557277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:29.569535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:49:29.574925 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:29.576905 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:29.578355 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 624/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:29.583189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb1816-925f-4059-803c-4f16d86363ad None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:29.716696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb1816-925f-4059-803c-4f16d86363ad None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:29.749580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:29.755571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:29.756256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:29.756564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:29.756834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:29.757497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:29.757795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:29.759196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:29.770957 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 626/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:29.777469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:29.877201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:29.915647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d63aa-6123-4a96-b30a-830c9988a6b7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:29.958927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:29.959760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:29.960129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:29.960372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:29.973085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:29.975295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:29.992092 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:29.993727 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:29.995209 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:29.995209 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:29.995209 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:29.995209 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 625/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:29.999826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:30.122918 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c67f75-87af-4342-9361-3e96159821cb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:30.162229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:30.162802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:30.163139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:30.163552 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:30.164412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:30.164741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:30.166520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:30.185049 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 627/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:30.196113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:30.319330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6414-e822-4dd0-9691-f3d6985571b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:30.355821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:30.367603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:30.368288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:30.368644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:30.369045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:30.380556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:30.382262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:49:30.395495 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:30.402636 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:30.403816 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 626/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:30.405205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:30.486374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:30.539464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e4052-ac03-429c-bdbb-dd75c85fcde9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:30.578379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:30.578998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:30.579275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:30.579549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:30.580280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:30.580607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:30.582035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:30.604066 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 628/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Sep 25 00:49:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:30.612763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:30.719988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65ce73-f586-42c5-8363-4d80e812f18a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:30.759680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:30.760636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:30.761627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:30.762055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:30.837623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:30.851944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:30.853460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:30.982377 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 627/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:49:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:30.988559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:31.113736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ff51d-de2d-4cf8-93d1-d5f7c84deb04 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:31.129829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:31.155099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:31.155735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:31.156018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:31.156333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:31.167352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:31.168854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:31.184826 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:31.186424 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:31.187636 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:31.187636 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:31.187636 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:31.190930 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 629/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:31.191867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:31.318679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:31.337111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe18a25-321a-4965-bc7a-c07bee315035 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:31.380310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:31.380984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:31.381260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:31.381659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:31.382484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:31.382834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:31.384473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:31.398826 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 628/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:31.407643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:31.529696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecbcb1-fd5f-4776-8043-b8548ecabb00 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:31.580551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:31.581645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:31.582137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:31.582618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:31.596113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:31.599731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:49:31.617875 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:31.619535 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:31.621225 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 630/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:49:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:49:31.630002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:31.783882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9560-6f2f-4f4e-b144-081cabc4c99f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:31.825750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da03f1-3fa4-4f98-ac81-e21d6702ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:31.826391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da03f1-3fa4-4f98-ac81-e21d6702ab43 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:31.826672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da03f1-3fa4-4f98-ac81-e21d6702ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:31.826946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da03f1-3fa4-4f98-ac81-e21d6702ab43 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:31.827675 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da03f1-3fa4-4f98-ac81-e21d6702ab43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:31.827984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da03f1-3fa4-4f98-ac81-e21d6702ab43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:31.829517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da03f1-3fa4-4f98-ac81-e21d6702ab43 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:31.844984 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 629/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:31.854178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:31.885467 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:31.974879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a41979-2667-4ac9-a04b-aa7f8fa40846 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:32.014458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07071fd-f216-41ae-851e-18b6247d6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:32.015072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07071fd-f216-41ae-851e-18b6247d6766 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:32.015354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07071fd-f216-41ae-851e-18b6247d6766 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:32.015625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07071fd-f216-41ae-851e-18b6247d6766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:32.016370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07071fd-f216-41ae-851e-18b6247d6766 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=d45df63e9f2449be8d13f58100775eb7) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:32.016703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07071fd-f216-41ae-851e-18b6247d6766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:32.018142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07071fd-f216-41ae-851e-18b6247d6766 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:32.036519 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 631/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:49:31 2022] GET /identity/v3/role_assignments?user.id=d45df63e9f2449be8d13f58100775eb7&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:34.294588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1998-b79e-4291-9426-076431879d23 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:34.295126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1998-b79e-4291-9426-076431879d23 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:34.295126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1998-b79e-4291-9426-076431879d23 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:34.296560 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 630/1261] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:34.301647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884207-5160-437a-ad29-3c49925802ee None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:34.302028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884207-5160-437a-ad29-3c49925802ee None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:34.302201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884207-5160-437a-ad29-3c49925802ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:34.322365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-21884207-5160-437a-ad29-3c49925802ee None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:34.324182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21884207-5160-437a-ad29-3c49925802ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:34.402620 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 632/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:34.919463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e1998-b79e-4291-9426-076431879d23 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:34.930652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:35.039060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e1998-b79e-4291-9426-076431879d23 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:35.075232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:35.075910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:35.076190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:35.076492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:35.086790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:49:35.092189 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:35.093597 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:35.094627 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 631/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:49:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:35.099972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884207-5160-437a-ad29-3c49925802ee None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:35.118250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:35.204307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884207-5160-437a-ad29-3c49925802ee None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:35.248357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:35.249249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:35.249631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:35.250142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:35.250995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:35.251440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:35.253171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:35.264954 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 633/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:35.270942 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:35.353549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:35.393642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edbf78-dfd4-4ec3-a51a-47e5bb5689b5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:35.433993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:35.434624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:35.434904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:35.435177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:35.446282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:35.447770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:35.464840 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:35.466648 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:35.468524 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:35.468524 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:35.468524 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:35.468524 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 632/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:35.472579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:35.557763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:35.578229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7793-4aee-454d-870d-06ffd0152423 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:35.612006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:35.612639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:35.612923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:35.613197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:35.613839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:35.614139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:35.615512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:35.628397 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 634/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:35.634298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:35.746310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5d446-33eb-40f2-9125-35a9939f3595 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:35.766366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:35.785845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:35.786483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:35.786770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:35.787041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:35.795150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:35.796659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:49:35.808728 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:35.809930 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:35.811031 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 633/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:35.816470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:35.921619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159911-b83b-4230-bc5e-ae2e7b103e39 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:35.955617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:35.956229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:35.956531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:35.956801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:35.957519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:35.957818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:35.959234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:35.968112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:35.977102 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 635/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Sep 25 00:49:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:35.983807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:36.094818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c80d-284c-4573-8e3e-feb4983f49b6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:36.133674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:36.134328 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:36.134615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:36.134897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:36.147138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:36.148629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:36.164245 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:36.165341 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:36.166257 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:36.166257 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:36.166257 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:36.166257 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 634/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:36.171856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:36.300036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847c83f-fadc-4c7e-872e-dd13b868a2fb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:36.354264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:36.354954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:36.355325 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:36.355588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:36.356599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:36.356911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:36.358895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:36.373650 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 636/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:36.382815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:36.418033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:36.491525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fd14d-43ad-4a6c-9331-3df947db3498 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:36.528350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:36.529043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:36.529326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:36.529600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:36.537690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:36.539287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:49:36.553406 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:36.555207 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:36.557024 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 635/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:36.561738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:36.633844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:36.679198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b96c0c-643c-4b8a-a6de-135bceb996b4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:36.713952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54919a5-39be-43dd-a930-df49693b80b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:36.714613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54919a5-39be-43dd-a930-df49693b80b6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:36.714893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54919a5-39be-43dd-a930-df49693b80b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:36.715191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54919a5-39be-43dd-a930-df49693b80b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:36.715967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54919a5-39be-43dd-a930-df49693b80b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:36.716276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54919a5-39be-43dd-a930-df49693b80b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:36.717822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54919a5-39be-43dd-a930-df49693b80b6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:36.737610 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 637/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:36.744140 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:36.845656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:36.856660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d224c-4377-487c-84ac-d722c0000392 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:36.896257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669631e-cc44-452c-897b-83f339ee0fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:36.896839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669631e-cc44-452c-897b-83f339ee0fbb None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:36.897121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669631e-cc44-452c-897b-83f339ee0fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:36.897397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669631e-cc44-452c-897b-83f339ee0fbb None admin] PATH_INFO: `/v3/projects/77f655eb34fc4559a31900ece5656154/users/d45df63e9f2449be8d13f58100775eb7/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:36.899610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669631e-cc44-452c-897b-83f339ee0fbb None admin] RBAC: Authorizing `identity:create_grant(project_id=77f655eb34fc4559a31900ece5656154, user_id=d45df63e9f2449be8d13f58100775eb7, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:36.901108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669631e-cc44-452c-897b-83f339ee0fbb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:36.917258 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 636/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Sun Sep 25 00:49:36 2022] PUT /identity/v3/projects/77f655eb34fc4559a31900ece5656154/users/d45df63e9f2449be8d13f58100775eb7/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:49:39.107771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19182b-0779-41bf-9383-6c61e05f5f9e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:39.108373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19182b-0779-41bf-9383-6c61e05f5f9e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:39.108575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19182b-0779-41bf-9383-6c61e05f5f9e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:39.109863 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 638/1274] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:39.116322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292176d6-3b28-40af-be4a-e8c6ea98e2d3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:39.116977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292176d6-3b28-40af-be4a-e8c6ea98e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:39.116977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292176d6-3b28-40af-be4a-e8c6ea98e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:39.138328 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-292176d6-3b28-40af-be4a-e8c6ea98e2d3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:39.139637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-292176d6-3b28-40af-be4a-e8c6ea98e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:39.233290 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 637/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:39.505891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19182b-0779-41bf-9383-6c61e05f5f9e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:39.641792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19182b-0779-41bf-9383-6c61e05f5f9e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:39.662930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:39.684373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:39.685031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:39.685311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:39.685583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:39.696917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 25 00:49:39.702229 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:39.703900 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:39.705354 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 639/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Sun Sep 25 00:49:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:39.710413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292176d6-3b28-40af-be4a-e8c6ea98e2d3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:39.802830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:39.823009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292176d6-3b28-40af-be4a-e8c6ea98e2d3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:39.858793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:39.859403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:39.859685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:39.859959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:39.860657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:39.860961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:39.862336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:39.870782 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 638/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Sun Sep 25 00:49:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:39.876935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:39.978637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8558-318d-4882-a25a-b1835de65abb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:40.013192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:40.013837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:40.014115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:40.014388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:40.025005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:40.026459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:40.041884 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:40.044117 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:40.045113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:40.045113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:40.045113 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:40.045113 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 640/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:40.049469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:40.171633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f8ad5-18e1-4f56-8eee-a0e038f13cd9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:40.211640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:40.212252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:40.212562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:40.212839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:40.213488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:40.213793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:40.215209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:40.227195 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 639/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:40.233121 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:40.269806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:40.351131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd0c53-bc17-4523-af0e-dda7b04da02d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:40.387556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:40.388191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:40.388506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:40.388783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:40.397871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:40.399607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 25 00:49:40.414171 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:40.415572 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:40.416819 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 641/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:40.420302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:40.424451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:40.528442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be994e-13cf-4974-a0d1-2fb605c860b9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:40.562323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:40.562937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:40.563220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:40.563497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:40.564233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:40.564697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:40.566141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:40.584777 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 640/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Sun Sep 25 00:49:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:40.591396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:40.690306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:40.701916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b3966-5af6-417d-9ff6-3a2917da82c5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:40.737292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:40.737896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:40.738177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:40.738454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:40.816530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:40.818051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:40.914811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:40.926592 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 642/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Sun Sep 25 00:49:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:40.932612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:40.996507 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:41.036509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b53ad2-7fa8-43ce-82e1-a36ef98314de None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:41.076791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:41.077430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:41.077708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:41.077979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:41.088670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:41.090153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 25 00:49:41.105517 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:41.107268 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 25 00:49:41.109318 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 25 00:49:41.109318 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 25 00:49:41.109318 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:41.109318 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 641/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:41.114023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:41.220454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87506274-64cf-4b52-8f5b-880a2cefdf67 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:41.255816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:41.257039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:41.257039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:41.257039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:41.257509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:41.257813 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:41.259173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:41.270688 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 643/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Sun Sep 25 00:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:41.276704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:41.392316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00948630-e6d1-47de-9198-d71c52e310af None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:41.406390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:41.427402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:41.428032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:41.428313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:41.428613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:41.435787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:41.437263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 25 00:49:41.448238 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:41.449375 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 25 00:49:41.450501 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 642/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Sun Sep 25 00:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:41.455327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:41.505293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:41.557660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688d276-bc29-4c7d-be2b-5e5e21bc6462 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:41.591658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392b1e4-9775-4b47-b62f-33477448867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:41.593373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392b1e4-9775-4b47-b62f-33477448867c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:41.595215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392b1e4-9775-4b47-b62f-33477448867c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:41.595507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392b1e4-9775-4b47-b62f-33477448867c None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:41.597430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392b1e4-9775-4b47-b62f-33477448867c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:41.597755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392b1e4-9775-4b47-b62f-33477448867c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:41.599207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392b1e4-9775-4b47-b62f-33477448867c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:41.612500 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 644/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Sun Sep 25 00:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:41.619030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:41.738118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504a0d7-326d-4c88-b58d-692507287495 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:41.778358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c318ee-7928-430b-a282-84cea7c43223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:41.778971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c318ee-7928-430b-a282-84cea7c43223 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:41.779251 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c318ee-7928-430b-a282-84cea7c43223 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:41.779604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c318ee-7928-430b-a282-84cea7c43223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:41.780486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c318ee-7928-430b-a282-84cea7c43223 None admin] RBAC: Adding query filter params (role.id=3e7f6c18b6fb41099cfa5861f5ae5278, scope.project.id=77f655eb34fc4559a31900ece5656154, user.id=d45df63e9f2449be8d13f58100775eb7) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:41.780797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c318ee-7928-430b-a282-84cea7c43223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:41.782241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c318ee-7928-430b-a282-84cea7c43223 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:41.788906 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 643/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Sun Sep 25 00:49:41 2022] GET /identity/v3/role_assignments?user.id=d45df63e9f2449be8d13f58100775eb7&scope.project.id=77f655eb34fc4559a31900ece5656154&role.id=3e7f6c18b6fb41099cfa5861f5ae5278 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 00:49:43.990568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b96495-25ee-4b0f-93ba-026348863e61 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:43.991133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b96495-25ee-4b0f-93ba-026348863e61 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:43.991133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b96495-25ee-4b0f-93ba-026348863e61 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:43.992488 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 645/1288] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:43.997868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5f0a-40ee-4be4-a0ba-3053367f365c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:43.998306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5f0a-40ee-4be4-a0ba-3053367f365c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:43.998409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5f0a-40ee-4be4-a0ba-3053367f365c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:44.016908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-590f5f0a-40ee-4be4-a0ba-3053367f365c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:44.018151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-590f5f0a-40ee-4be4-a0ba-3053367f365c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:44.052630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:44.092979 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 644/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:44.366893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b96495-25ee-4b0f-93ba-026348863e61 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:44.468399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b96495-25ee-4b0f-93ba-026348863e61 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:44.479959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:44.504812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:44.505457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:44.505734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:44.506006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:44.506648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:44.507954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:44.518908 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:44.520457 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:44.521607 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 646/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:49:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 25 00:49:44.528256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f5f0a-40ee-4be4-a0ba-3053367f365c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:44.628121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f5f0a-40ee-4be4-a0ba-3053367f365c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:44.663014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be47a-9620-4f22-8428-dc6d4cc92b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:44.663626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be47a-9620-4f22-8428-dc6d4cc92b4a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:44.663907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be47a-9620-4f22-8428-dc6d4cc92b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:44.664186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be47a-9620-4f22-8428-dc6d4cc92b4a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:44.664854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be47a-9620-4f22-8428-dc6d4cc92b4a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:44.665157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be47a-9620-4f22-8428-dc6d4cc92b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:44.666532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be47a-9620-4f22-8428-dc6d4cc92b4a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:44.675757 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 645/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Sep 25 00:49:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:44.681671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:44.782215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c6b96-58e9-4582-b12a-7f4e20061a65 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:44.818657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04ba61-6cfa-4470-86a3-bb9006e0508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:44.819325 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04ba61-6cfa-4470-86a3-bb9006e0508c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:44.819606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04ba61-6cfa-4470-86a3-bb9006e0508c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:44.819879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04ba61-6cfa-4470-86a3-bb9006e0508c None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:44.820545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04ba61-6cfa-4470-86a3-bb9006e0508c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:44.820851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04ba61-6cfa-4470-86a3-bb9006e0508c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:44.822216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee04ba61-6cfa-4470-86a3-bb9006e0508c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:44.828918 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 647/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Sep 25 00:49:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:47.061797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b7364-9244-4f85-a8bc-faf6ae618144 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:47.062447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b7364-9244-4f85-a8bc-faf6ae618144 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:47.062642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b7364-9244-4f85-a8bc-faf6ae618144 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:47.065029 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 646/1293] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:47.074390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3df63-715c-4091-92ae-ca4fc20e7d31 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:47.075196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3df63-715c-4091-92ae-ca4fc20e7d31 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:47.075326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3df63-715c-4091-92ae-ca4fc20e7d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:47.107935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fbe3df63-715c-4091-92ae-ca4fc20e7d31 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:47.109203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fbe3df63-715c-4091-92ae-ca4fc20e7d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:47.150484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:47.201330 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 648/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:49:47.496019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b7364-9244-4f85-a8bc-faf6ae618144 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:47.533051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:47.621129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b7364-9244-4f85-a8bc-faf6ae618144 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:47.659317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec9331-48a5-48a0-b683-4aa583cd6b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:47.659929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9331-48a5-48a0-b683-4aa583cd6b69 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:47.660209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9331-48a5-48a0-b683-4aa583cd6b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:47.660513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec9331-48a5-48a0-b683-4aa583cd6b69 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:47.661184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec9331-48a5-48a0-b683-4aa583cd6b69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:47.662582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec9331-48a5-48a0-b683-4aa583cd6b69 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:47.680480 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 647/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Sun Sep 25 00:49:47 2022] POST /identity/v3/services => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 00:49:50.793058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da166be0-726a-4c71-8581-c4dae960c98f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:50.793924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da166be0-726a-4c71-8581-c4dae960c98f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:50.793924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da166be0-726a-4c71-8581-c4dae960c98f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:50.795307 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 649/1296] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:49:50.801544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a8019-2897-44f6-8178-dbce77713b84 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:50.802355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a8019-2897-44f6-8178-dbce77713b84 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:50.802355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a8019-2897-44f6-8178-dbce77713b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:50.822841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9b7a8019-2897-44f6-8178-dbce77713b84 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:50.824905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b7a8019-2897-44f6-8178-dbce77713b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:50.939859 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 648/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 25 00:49:51.451666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da166be0-726a-4c71-8581-c4dae960c98f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:51.603289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da166be0-726a-4c71-8581-c4dae960c98f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:51.644776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:51.645415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:51.645784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:51.646083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:51.646736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:51.648129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:51.657735 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:51.659590 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:51.660969 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 650/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:49:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:49:51.668991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a8019-2897-44f6-8178-dbce77713b84 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:51.688233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:51.775217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a8019-2897-44f6-8178-dbce77713b84 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:51.812945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:51.813678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:51.813879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:51.814146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:51.814787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:51.815084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:51.816511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:51.822291 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 649/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Sep 25 00:49:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:51.832131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:51.874904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:51.959233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96b27e-5ea5-4d96-ab57-f1e3ba215f92 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:52.003641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b6d1f-d322-42b4-8abf-4c3a08357b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:52.004802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b6d1f-d322-42b4-8abf-4c3a08357b10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:52.005214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b6d1f-d322-42b4-8abf-4c3a08357b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:52.005630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b6d1f-d322-42b4-8abf-4c3a08357b10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:52.006723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b6d1f-d322-42b4-8abf-4c3a08357b10 None admin] RBAC: Adding query filter params (interface=public, service_id=3d23e254394a4060997a31b4d40846f0, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:52.007259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b6d1f-d322-42b4-8abf-4c3a08357b10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:52.009399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b6d1f-d322-42b4-8abf-4c3a08357b10 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:52.016964 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 651/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Sun Sep 25 00:49:51 2022] GET /identity/v3/endpoints?service_id=3d23e254394a4060997a31b4d40846f0&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:52.023733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:52.128674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:52.148243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc437dbd-4138-4837-bd8e-57ab70bf219a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:52.192079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91498d0a-ebc8-423c-a31d-9dbd2d75c4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:52.192713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91498d0a-ebc8-423c-a31d-9dbd2d75c4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:52.193037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91498d0a-ebc8-423c-a31d-9dbd2d75c4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:52.193258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91498d0a-ebc8-423c-a31d-9dbd2d75c4f9 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:52.193863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91498d0a-ebc8-423c-a31d-9dbd2d75c4f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:52.195245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91498d0a-ebc8-423c-a31d-9dbd2d75c4f9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:52.202952 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 650/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Sun Sep 25 00:49:52 2022] GET /identity/v3/services => generated 2191 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 00:49:54.516472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8dd5-9744-42f8-b16f-8a7bd4253ed4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:54.517192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8dd5-9744-42f8-b16f-8a7bd4253ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:54.517192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c8dd5-9744-42f8-b16f-8a7bd4253ed4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:54.518438 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 652/1302] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:49:54.524840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169634-a415-43a0-a9bd-ba7cf37322e0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:54.525607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169634-a415-43a0-a9bd-ba7cf37322e0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:54.525607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b169634-a415-43a0-a9bd-ba7cf37322e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:54.546215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7b169634-a415-43a0-a9bd-ba7cf37322e0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:49:54.547435 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7b169634-a415-43a0-a9bd-ba7cf37322e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:49:54.629754 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 651/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:49:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 25 00:49:54.985382 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c8dd5-9744-42f8-b16f-8a7bd4253ed4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:55.114050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:55.128732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c8dd5-9744-42f8-b16f-8a7bd4253ed4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:55.185649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:55.186614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:55.187050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:55.187331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:55.188172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:55.189879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 25 00:49:55.202578 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 25 00:49:55.204315 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 25 00:49:55.206139 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 653/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Sun Sep 25 00:49:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 25 00:49:55.213301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b169634-a415-43a0-a9bd-ba7cf37322e0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:55.336020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:55.379516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b169634-a415-43a0-a9bd-ba7cf37322e0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:55.441296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d7988-fbcf-44a9-a7e0-3ce65855a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:55.442234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7988-fbcf-44a9-a7e0-3ce65855a2ad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:55.442659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7988-fbcf-44a9-a7e0-3ce65855a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:55.443100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7988-fbcf-44a9-a7e0-3ce65855a2ad None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:55.444141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7988-fbcf-44a9-a7e0-3ce65855a2ad None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:49:55.444637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7988-fbcf-44a9-a7e0-3ce65855a2ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:55.446961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7988-fbcf-44a9-a7e0-3ce65855a2ad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:55.457009 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 652/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Sun Sep 25 00:49:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:49:55.464454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:49:55.622463 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24914c88-c6ae-4b05-9a11-acc3df88f191 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:49:55.670219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f85b42-963c-41d0-8357-9031d73f00f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:49:55.670939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f85b42-963c-41d0-8357-9031d73f00f8 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:49:55.671256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f85b42-963c-41d0-8357-9031d73f00f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:49:55.671580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f85b42-963c-41d0-8357-9031d73f00f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:49:55.672330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f85b42-963c-41d0-8357-9031d73f00f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:49:55.673853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f85b42-963c-41d0-8357-9031d73f00f8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:49:55.702274 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 654/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:49:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:50:05.451129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=30 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:50:05.695193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:15.421732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423213-c4e1-48ca-ab36-2dabdd75c633 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:15.422748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423213-c4e1-48ca-ab36-2dabdd75c633 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:15.423175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423213-c4e1-48ca-ab36-2dabdd75c633 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:15.430073 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 653/1307] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:15.432528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7c98-95fd-40cd-b64d-ff8c31f72216 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:15.434061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7c98-95fd-40cd-b64d-ff8c31f72216 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:15.434856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7c98-95fd-40cd-b64d-ff8c31f72216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:15.604091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-626e7c98-95fd-40cd-b64d-ff8c31f72216 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:15.606013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-626e7c98-95fd-40cd-b64d-ff8c31f72216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:15.776845 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 655/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:16.094754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903ed9-b709-41fe-9db6-f91be01b863c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:16.095038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903ed9-b709-41fe-9db6-f91be01b863c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:16.095611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903ed9-b709-41fe-9db6-f91be01b863c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:16.097278 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 654/1309] 10.222.0.26 () {54 vars in 964 bytes} [Sun Sep 25 00:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:16.105438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d9470-554b-48d6-aa74-63ded6ee4661 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:16.105916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d9470-554b-48d6-aa74-63ded6ee4661 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:16.106300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d9470-554b-48d6-aa74-63ded6ee4661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:16.257492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c05d9470-554b-48d6-aa74-63ded6ee4661 None None] MFA Rules not processed for user `8427b2d4e8c040d0a1b00e25e3b58e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:16.258715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c05d9470-554b-48d6-aa74-63ded6ee4661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:16.283599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:16.361511 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 656/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Sun Sep 25 00:51:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:51:16.371468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59903ed9-b709-41fe-9db6-f91be01b863c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:16.373320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-59903ed9-b709-41fe-9db6-f91be01b863c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:16.504798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59903ed9-b709-41fe-9db6-f91be01b863c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:16.614097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd91359-1e2c-4755-aee1-a0418d6d35ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:16.614779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd91359-1e2c-4755-aee1-a0418d6d35ff service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:16.615061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd91359-1e2c-4755-aee1-a0418d6d35ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:16.615342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd91359-1e2c-4755-aee1-a0418d6d35ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:16.617030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6bd91359-1e2c-4755-aee1-a0418d6d35ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:16.689897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:16.770371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd91359-1e2c-4755-aee1-a0418d6d35ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:16.773177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd91359-1e2c-4755-aee1-a0418d6d35ff service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:16.799935 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 655/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:19.615245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535c96a-48eb-440f-9145-ce0b1dea16aa None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:19.616061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535c96a-48eb-440f-9145-ce0b1dea16aa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:19.616061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535c96a-48eb-440f-9145-ce0b1dea16aa None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:19.617263 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 657/1312] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:19.623078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72ab5-3606-401a-899a-0d63db226765 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:19.623895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72ab5-3606-401a-899a-0d63db226765 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:19.623895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72ab5-3606-401a-899a-0d63db226765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:19.730004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f1f72ab5-3606-401a-899a-0d63db226765 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:19.732117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f1f72ab5-3606-401a-899a-0d63db226765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:19.925013 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 656/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:20.276442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6a8f-c492-4b90-b719-949145afcd53 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:20.276891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6a8f-c492-4b90-b719-949145afcd53 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:20.277252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6a8f-c492-4b90-b719-949145afcd53 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:20.278705 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 658/1314] 10.222.0.26 () {54 vars in 964 bytes} [Sun Sep 25 00:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:20.286848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460bf8c-c57c-4d40-a212-72d37c181cde None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:20.287444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460bf8c-c57c-4d40-a212-72d37c181cde None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:20.287895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460bf8c-c57c-4d40-a212-72d37c181cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:20.308411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:20.411339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2460bf8c-c57c-4d40-a212-72d37c181cde None None] MFA Rules not processed for user `8427b2d4e8c040d0a1b00e25e3b58e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:20.412710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2460bf8c-c57c-4d40-a212-72d37c181cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:20.520095 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 657/1315] 10.222.0.26 () {58 vars in 1061 bytes} [Sun Sep 25 00:51:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:20.535220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6a8f-c492-4b90-b719-949145afcd53 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:20.537579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c1af6a8f-c492-4b90-b719-949145afcd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:20.686094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6a8f-c492-4b90-b719-949145afcd53 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:20.785699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:20.789805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ec4c8-576b-4cc2-9a6a-11f7b39978be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:20.790713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ec4c8-576b-4cc2-9a6a-11f7b39978be service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:20.791219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ec4c8-576b-4cc2-9a6a-11f7b39978be service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:20.791670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ec4c8-576b-4cc2-9a6a-11f7b39978be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:20.793798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c2ec4c8-576b-4cc2-9a6a-11f7b39978be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:20.956977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ec4c8-576b-4cc2-9a6a-11f7b39978be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:20.960305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ec4c8-576b-4cc2-9a6a-11f7b39978be service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:20.985420 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 659/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:24.019427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7159ac-5e76-4217-a8bd-7be025694abb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:24.020038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7159ac-5e76-4217-a8bd-7be025694abb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:24.020038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7159ac-5e76-4217-a8bd-7be025694abb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:24.022167 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 658/1317] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:51:24.028270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04656fa3-a1af-491e-932a-38fe2e6e1060 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:24.028942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04656fa3-a1af-491e-932a-38fe2e6e1060 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:24.028942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04656fa3-a1af-491e-932a-38fe2e6e1060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:24.123789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-04656fa3-a1af-491e-932a-38fe2e6e1060 None None] MFA Rules not processed for user `f8ff084606084f0fb810529a2bea08e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:24.125192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-04656fa3-a1af-491e-932a-38fe2e6e1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:24.196320 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:24.260434 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 660/1318] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:51:24.568822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7159ac-5e76-4217-a8bd-7be025694abb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:24.571609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6a7159ac-5e76-4217-a8bd-7be025694abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:24.749214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7159ac-5e76-4217-a8bd-7be025694abb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:24.752702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:24.853555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc74f22-6982-44e6-85bc-e0ef2984fb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:24.854263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc74f22-6982-44e6-85bc-e0ef2984fb20 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:24.854689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc74f22-6982-44e6-85bc-e0ef2984fb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:24.855064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc74f22-6982-44e6-85bc-e0ef2984fb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:24.856974 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cc74f22-6982-44e6-85bc-e0ef2984fb20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:24.991299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc74f22-6982-44e6-85bc-e0ef2984fb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:24.992840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc74f22-6982-44e6-85bc-e0ef2984fb20 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:25.014040 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 659/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:28.376632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669dd56-ba48-473f-8f92-6b10a0535b8c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:28.377491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669dd56-ba48-473f-8f92-6b10a0535b8c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:28.377655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669dd56-ba48-473f-8f92-6b10a0535b8c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:28.380289 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 661/1320] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:28.386893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95c94-984c-4e1e-b173-6cc82cb825c5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:28.387457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95c94-984c-4e1e-b173-6cc82cb825c5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:28.387875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95c94-984c-4e1e-b173-6cc82cb825c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:28.461900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:28.502841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5ad95c94-984c-4e1e-b173-6cc82cb825c5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:28.504953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5ad95c94-984c-4e1e-b173-6cc82cb825c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:28.669429 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 660/1321] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:29.205585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669dd56-ba48-473f-8f92-6b10a0535b8c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:29.208333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7669dd56-ba48-473f-8f92-6b10a0535b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:29.400819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669dd56-ba48-473f-8f92-6b10a0535b8c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:29.515413 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba664e8-4fcc-4b90-99fb-79e6f30062e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:29.516333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba664e8-4fcc-4b90-99fb-79e6f30062e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:29.516843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba664e8-4fcc-4b90-99fb-79e6f30062e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:29.517321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba664e8-4fcc-4b90-99fb-79e6f30062e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:29.520555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dba664e8-4fcc-4b90-99fb-79e6f30062e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:29.534717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:29.712337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba664e8-4fcc-4b90-99fb-79e6f30062e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:29.714390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba664e8-4fcc-4b90-99fb-79e6f30062e9 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:29.748904 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 662/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:33.879852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fd6e8-ace6-476b-9fcd-d44dc4311591 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:33.880523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fd6e8-ace6-476b-9fcd-d44dc4311591 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:33.880863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fd6e8-ace6-476b-9fcd-d44dc4311591 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:33.882371 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 661/1323] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:33.892022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5da432-7b71-4b58-bb0c-dcfa998e0299 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:33.892743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5da432-7b71-4b58-bb0c-dcfa998e0299 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:33.893028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5da432-7b71-4b58-bb0c-dcfa998e0299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:34.002924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4a5da432-7b71-4b58-bb0c-dcfa998e0299 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:34.005045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4a5da432-7b71-4b58-bb0c-dcfa998e0299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:34.087994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:34.161372 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 663/1324] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:34.562606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fd6e8-ace6-476b-9fcd-d44dc4311591 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:34.564546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-689fd6e8-ace6-476b-9fcd-d44dc4311591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:34.693530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fd6e8-ace6-476b-9fcd-d44dc4311591 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:34.713880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:34.795085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b4bf6-8322-4fd8-8d45-afacd9f6aedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:34.795754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4bf6-8322-4fd8-8d45-afacd9f6aedd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:34.796032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4bf6-8322-4fd8-8d45-afacd9f6aedd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:34.796302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4bf6-8322-4fd8-8d45-afacd9f6aedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:34.797960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e1b4bf6-8322-4fd8-8d45-afacd9f6aedd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:34.931168 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b4bf6-8322-4fd8-8d45-afacd9f6aedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:34.932659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b4bf6-8322-4fd8-8d45-afacd9f6aedd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:34.952814 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 662/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:38.373549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3de2b-ba82-49e7-8cdf-6a84e32a2777 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:38.374430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3de2b-ba82-49e7-8cdf-6a84e32a2777 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:38.374430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3de2b-ba82-49e7-8cdf-6a84e32a2777 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:38.376329 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 664/1326] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:38.383915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb266-902d-45c7-a8ac-754ef2db7463 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:38.384557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb266-902d-45c7-a8ac-754ef2db7463 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:38.385036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb266-902d-45c7-a8ac-754ef2db7463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:38.482084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:38.516566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2acdb266-902d-45c7-a8ac-754ef2db7463 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:38.518603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2acdb266-902d-45c7-a8ac-754ef2db7463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:38.715943 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 663/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:39.255856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3de2b-ba82-49e7-8cdf-6a84e32a2777 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:39.258683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8fa3de2b-ba82-49e7-8cdf-6a84e32a2777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:39.447258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3de2b-ba82-49e7-8cdf-6a84e32a2777 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:39.552718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b165e-4023-43c1-aca7-54c13a289e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:39.553344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b165e-4023-43c1-aca7-54c13a289e27 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:39.553626 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b165e-4023-43c1-aca7-54c13a289e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:39.553899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b165e-4023-43c1-aca7-54c13a289e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:39.555572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-639b165e-4023-43c1-aca7-54c13a289e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:39.577903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:39.709074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b165e-4023-43c1-aca7-54c13a289e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:39.710633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b165e-4023-43c1-aca7-54c13a289e27 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:39.734395 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 665/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:42.324450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e00ae-c978-4874-acf3-20da8e51d428 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:42.325194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e00ae-c978-4874-acf3-20da8e51d428 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:42.325467 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e00ae-c978-4874-acf3-20da8e51d428 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:42.327985 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 664/1329] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:42.338445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac592c-cd69-4d57-8668-98448b06141f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:42.338974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac592c-cd69-4d57-8668-98448b06141f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:42.338974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac592c-cd69-4d57-8668-98448b06141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:42.420802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-96ac592c-cd69-4d57-8668-98448b06141f None None] MFA Rules not processed for user `f8ff084606084f0fb810529a2bea08e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:42.422058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-96ac592c-cd69-4d57-8668-98448b06141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:42.503444 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:42.556353 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 666/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:42.860149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e00ae-c978-4874-acf3-20da8e51d428 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:42.861743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b81e00ae-c978-4874-acf3-20da8e51d428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:42.999879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e00ae-c978-4874-acf3-20da8e51d428 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:43.024283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:43.083114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352abb6-bc03-48f0-bb45-8bcda8272ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:43.083754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352abb6-bc03-48f0-bb45-8bcda8272ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:43.084029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352abb6-bc03-48f0-bb45-8bcda8272ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:43.084345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352abb6-bc03-48f0-bb45-8bcda8272ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:43.086113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2352abb6-bc03-48f0-bb45-8bcda8272ca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:43.248770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352abb6-bc03-48f0-bb45-8bcda8272ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:43.251506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352abb6-bc03-48f0-bb45-8bcda8272ca2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:43.286195 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 665/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:45.935051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f264208-4a24-4f07-b608-d6e8e36d7486 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:45.935604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f264208-4a24-4f07-b608-d6e8e36d7486 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:45.935604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f264208-4a24-4f07-b608-d6e8e36d7486 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:45.937000 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 667/1332] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:45.943088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a602-d2f8-47fa-a917-23ab8cb30c07 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:45.943647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a602-d2f8-47fa-a917-23ab8cb30c07 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:45.943647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a602-d2f8-47fa-a917-23ab8cb30c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:46.021662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:46.025605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d099a602-d2f8-47fa-a917-23ab8cb30c07 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:46.027357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d099a602-d2f8-47fa-a917-23ab8cb30c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:46.145698 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 666/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:46.448332 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f264208-4a24-4f07-b608-d6e8e36d7486 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:46.450017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7f264208-4a24-4f07-b608-d6e8e36d7486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:46.583720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f264208-4a24-4f07-b608-d6e8e36d7486 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:46.671683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94c5bc-c1bc-4593-899d-9bb058059ba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:46.672293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94c5bc-c1bc-4593-899d-9bb058059ba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:46.672609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94c5bc-c1bc-4593-899d-9bb058059ba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:46.672883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94c5bc-c1bc-4593-899d-9bb058059ba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:46.674503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ef94c5bc-c1bc-4593-899d-9bb058059ba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:46.703723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:46.812646 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94c5bc-c1bc-4593-899d-9bb058059ba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:46.814204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94c5bc-c1bc-4593-899d-9bb058059ba7 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:46.835826 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 668/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:49.244860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631cc1-2d87-42ec-8f00-05e1e1588811 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:49.245748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631cc1-2d87-42ec-8f00-05e1e1588811 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:49.245748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631cc1-2d87-42ec-8f00-05e1e1588811 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:49.247459 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 667/1335] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:49.254691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cee02-f1e2-423f-a947-6fa1ae983b24 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:49.255282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cee02-f1e2-423f-a947-6fa1ae983b24 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:49.255657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cee02-f1e2-423f-a947-6fa1ae983b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:49.369819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-523cee02-f1e2-423f-a947-6fa1ae983b24 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:49.372015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-523cee02-f1e2-423f-a947-6fa1ae983b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:49.498669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:49.528617 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 669/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:50.075617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b631cc1-2d87-42ec-8f00-05e1e1588811 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:50.077808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7b631cc1-2d87-42ec-8f00-05e1e1588811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:50.208051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b631cc1-2d87-42ec-8f00-05e1e1588811 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:50.243599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:50.299415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd9a3f-7ffc-4c0c-b9a8-5dfdafba4a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:50.301181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd9a3f-7ffc-4c0c-b9a8-5dfdafba4a46 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:50.301181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd9a3f-7ffc-4c0c-b9a8-5dfdafba4a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:50.301181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd9a3f-7ffc-4c0c-b9a8-5dfdafba4a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:50.302872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80fd9a3f-7ffc-4c0c-b9a8-5dfdafba4a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:50.495714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd9a3f-7ffc-4c0c-b9a8-5dfdafba4a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:50.498609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd9a3f-7ffc-4c0c-b9a8-5dfdafba4a46 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:50.524562 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 668/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:51:53.481706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c748dc-94df-4dca-a4a5-38fed55a809a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:53.482535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c748dc-94df-4dca-a4a5-38fed55a809a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:53.482535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c748dc-94df-4dca-a4a5-38fed55a809a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:53.483836 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 670/1338] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:51:53.494192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b5e75-b27b-4243-8c7b-eb63b2dc6ffd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:53.495081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b5e75-b27b-4243-8c7b-eb63b2dc6ffd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:53.495244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b5e75-b27b-4243-8c7b-eb63b2dc6ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:53.630728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-242b5e75-b27b-4243-8c7b-eb63b2dc6ffd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:51:53.632936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-242b5e75-b27b-4243-8c7b-eb63b2dc6ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:53.650485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:53.798233 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 669/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:51:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:51:54.339540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c748dc-94df-4dca-a4a5-38fed55a809a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:51:54.341372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f0c748dc-94df-4dca-a4a5-38fed55a809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:54.505864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c748dc-94df-4dca-a4a5-38fed55a809a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:51:54.651822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c35240-902d-49b2-9b17-eab0dbaab720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:51:54.652808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c35240-902d-49b2-9b17-eab0dbaab720 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:51:54.653295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c35240-902d-49b2-9b17-eab0dbaab720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:51:54.653769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c35240-902d-49b2-9b17-eab0dbaab720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:51:54.656981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f7c35240-902d-49b2-9b17-eab0dbaab720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:51:54.713450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:51:54.838406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c35240-902d-49b2-9b17-eab0dbaab720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:51:54.841348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c35240-902d-49b2-9b17-eab0dbaab720 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:51:54.878660 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 671/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:51:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:52:00.885198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc47ce-e316-4b75-9705-02fd7022db9b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:00.886018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc47ce-e316-4b75-9705-02fd7022db9b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:00.886018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc47ce-e316-4b75-9705-02fd7022db9b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:00.888981 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 670/1341] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:52:00.894660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd6ab-edc8-45c5-916e-78cac01b1d0f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:00.895713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd6ab-edc8-45c5-916e-78cac01b1d0f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:00.895713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd6ab-edc8-45c5-916e-78cac01b1d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:01.026997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c7efd6ab-edc8-45c5-916e-78cac01b1d0f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:01.030072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c7efd6ab-edc8-45c5-916e-78cac01b1d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:01.211539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:01.234012 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 672/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:01.573701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc47ce-e316-4b75-9705-02fd7022db9b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:01.575600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c8dc47ce-e316-4b75-9705-02fd7022db9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:01.754204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc47ce-e316-4b75-9705-02fd7022db9b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:01.805912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:01.859417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfdaff-2fdf-4c3d-a3ba-afa4c82b70a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:01.860183 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfdaff-2fdf-4c3d-a3ba-afa4c82b70a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:01.860593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfdaff-2fdf-4c3d-a3ba-afa4c82b70a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:01.860748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfdaff-2fdf-4c3d-a3ba-afa4c82b70a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:01.863006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8abfdaff-2fdf-4c3d-a3ba-afa4c82b70a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:02.020989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfdaff-2fdf-4c3d-a3ba-afa4c82b70a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:02.023832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abfdaff-2fdf-4c3d-a3ba-afa4c82b70a5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:02.051987 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 671/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:52:06.361509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed085e-be80-46e1-a728-548263fbc19c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:06.362272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed085e-be80-46e1-a728-548263fbc19c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:06.362404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed085e-be80-46e1-a728-548263fbc19c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:06.364643 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 673/1344] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:52:06.371788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bddf0a-afd9-40b8-acb2-5e90ef87cca7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:06.371788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bddf0a-afd9-40b8-acb2-5e90ef87cca7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:06.371788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bddf0a-afd9-40b8-acb2-5e90ef87cca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:06.503577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f1bddf0a-afd9-40b8-acb2-5e90ef87cca7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:06.504898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f1bddf0a-afd9-40b8-acb2-5e90ef87cca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:06.528178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:06.638480 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 672/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:06.948191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed085e-be80-46e1-a728-548263fbc19c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:06.951564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8bed085e-be80-46e1-a728-548263fbc19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:07.153472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed085e-be80-46e1-a728-548263fbc19c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:07.260321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafa92a-026f-4efb-8801-734fb52cda76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:07.260938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa92a-026f-4efb-8801-734fb52cda76 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:07.261222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa92a-026f-4efb-8801-734fb52cda76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:07.261492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafa92a-026f-4efb-8801-734fb52cda76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:07.263180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-adafa92a-026f-4efb-8801-734fb52cda76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:07.358294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:07.471712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafa92a-026f-4efb-8801-734fb52cda76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:07.473344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafa92a-026f-4efb-8801-734fb52cda76 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:07.496415 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 674/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:52:13.421199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233c3b4-46db-476a-b2f3-4519a1327896 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:13.422071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233c3b4-46db-476a-b2f3-4519a1327896 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:13.422241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233c3b4-46db-476a-b2f3-4519a1327896 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:13.425233 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 673/1347] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:13.434583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363eb1e-fb64-4abe-bc9d-771691483242 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:13.435372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363eb1e-fb64-4abe-bc9d-771691483242 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:13.435514 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363eb1e-fb64-4abe-bc9d-771691483242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:13.550640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f363eb1e-fb64-4abe-bc9d-771691483242 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:13.552099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f363eb1e-fb64-4abe-bc9d-771691483242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:13.700229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:13.705294 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 675/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:14.016706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233c3b4-46db-476a-b2f3-4519a1327896 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:14.019799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b233c3b4-46db-476a-b2f3-4519a1327896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:14.171197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233c3b4-46db-476a-b2f3-4519a1327896 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:14.237384 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:14.281898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85167738-443e-4c24-9ed0-49f4e7a8b715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:14.282587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167738-443e-4c24-9ed0-49f4e7a8b715 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:14.283440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167738-443e-4c24-9ed0-49f4e7a8b715 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:14.283870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85167738-443e-4c24-9ed0-49f4e7a8b715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:14.286306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85167738-443e-4c24-9ed0-49f4e7a8b715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:14.427243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85167738-443e-4c24-9ed0-49f4e7a8b715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:14.428840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85167738-443e-4c24-9ed0-49f4e7a8b715 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:14.451082 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 674/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:52:20.344557 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546b85e-c481-429a-84f6-a3f67e65090e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:20.345191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546b85e-c481-429a-84f6-a3f67e65090e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:20.345820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546b85e-c481-429a-84f6-a3f67e65090e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:20.347623 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 676/1350] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 00:52:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:20.354443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de75e-4dbf-4784-afdc-b9c3ac6030ec None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:20.354443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de75e-4dbf-4784-afdc-b9c3ac6030ec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:20.354443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5de75e-4dbf-4784-afdc-b9c3ac6030ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:20.486560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4a5de75e-4dbf-4784-afdc-b9c3ac6030ec None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:20.488365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4a5de75e-4dbf-4784-afdc-b9c3ac6030ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:20.514974 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:20.698172 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 675/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 00:52:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:52:26.209535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df954ddc-7f19-425b-8094-948e127e0516 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:26.210875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df954ddc-7f19-425b-8094-948e127e0516 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:26.211390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df954ddc-7f19-425b-8094-948e127e0516 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:26.213474 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 677/1352] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 00:52:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:26.221718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e437bc-af9b-4bde-beb2-e633ac09a219 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:26.222604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e437bc-af9b-4bde-beb2-e633ac09a219 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:26.222604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e437bc-af9b-4bde-beb2-e633ac09a219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:26.362307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-67e437bc-af9b-4bde-beb2-e633ac09a219 None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:26.363874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-67e437bc-af9b-4bde-beb2-e633ac09a219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:26.511308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:26.576815 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 676/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 00:52:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:28.187114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2a41-148d-4c7e-bb37-471093a48a24 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:28.187970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2a41-148d-4c7e-bb37-471093a48a24 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:28.187970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2a41-148d-4c7e-bb37-471093a48a24 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:28.190741 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 678/1354] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 00:52:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:28.197318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e4f91-4d7e-479a-96a1-cfcf6450463f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:28.197318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e4f91-4d7e-479a-96a1-cfcf6450463f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:28.197318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e4f91-4d7e-479a-96a1-cfcf6450463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:28.303309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-000e4f91-4d7e-479a-96a1-cfcf6450463f None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:28.307458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-000e4f91-4d7e-479a-96a1-cfcf6450463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:28.472734 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 677/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 00:52:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:52:33.198339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170061f-144d-402f-9974-868862c44ecf None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:33.198339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170061f-144d-402f-9974-868862c44ecf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:33.199035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170061f-144d-402f-9974-868862c44ecf None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:33.201150 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 679/1356] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:33.209962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde5cb9-eb98-43e5-8b80-13839dc0efb5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:33.211009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde5cb9-eb98-43e5-8b80-13839dc0efb5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:33.211294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde5cb9-eb98-43e5-8b80-13839dc0efb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:33.262468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:33.315315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0fde5cb9-eb98-43e5-8b80-13839dc0efb5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:33.317795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0fde5cb9-eb98-43e5-8b80-13839dc0efb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:33.463456 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 678/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:52:33.797789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772145e-44b8-4a15-98a5-4ab164b31028 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:33.798180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772145e-44b8-4a15-98a5-4ab164b31028 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:33.798875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772145e-44b8-4a15-98a5-4ab164b31028 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:33.801731 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 680/1358] 10.222.0.26 () {54 vars in 961 bytes} [Sun Sep 25 00:52:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:33.810848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda377b5-407e-4224-8bbc-f1e4e58e279e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:33.811169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda377b5-407e-4224-8bbc-f1e4e58e279e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:33.811398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda377b5-407e-4224-8bbc-f1e4e58e279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:33.953632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fda377b5-407e-4224-8bbc-f1e4e58e279e None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:33.954801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fda377b5-407e-4224-8bbc-f1e4e58e279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:34.090388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:34.230050 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 679/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Sep 25 00:52:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:52:34.244542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772145e-44b8-4a15-98a5-4ab164b31028 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:34.247328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a772145e-44b8-4a15-98a5-4ab164b31028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:34.553816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772145e-44b8-4a15-98a5-4ab164b31028 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:34.712598 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:34.805389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef7dfd-362e-4164-b986-849c9f4216c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:34.806473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7dfd-362e-4164-b986-849c9f4216c6 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:34.807076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7dfd-362e-4164-b986-849c9f4216c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:34.807670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7dfd-362e-4164-b986-849c9f4216c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:34.811039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eaef7dfd-362e-4164-b986-849c9f4216c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:35.010369 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef7dfd-362e-4164-b986-849c9f4216c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:35.014144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef7dfd-362e-4164-b986-849c9f4216c6 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:35.054495 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 681/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 811 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:52:38.773485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcd65a-2862-4ef8-a332-ce3e355bc601 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:38.774151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcd65a-2862-4ef8-a332-ce3e355bc601 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:38.774465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcd65a-2862-4ef8-a332-ce3e355bc601 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:38.777053 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 680/1361] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:38.784247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cce93-a765-4b93-8b44-abb4c5b8644f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:38.785096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cce93-a765-4b93-8b44-abb4c5b8644f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:38.785096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cce93-a765-4b93-8b44-abb4c5b8644f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:38.896620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-060cce93-a765-4b93-8b44-abb4c5b8644f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:38.897983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-060cce93-a765-4b93-8b44-abb4c5b8644f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:38.954754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:39.055987 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 682/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:39.409170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcd65a-2862-4ef8-a332-ce3e355bc601 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:39.410713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e9bcd65a-2862-4ef8-a332-ce3e355bc601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:39.696769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcd65a-2862-4ef8-a332-ce3e355bc601 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:39.717388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:39.877471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4ecce-4d67-48a7-8cac-be609fe82f5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:39.878439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4ecce-4d67-48a7-8cac-be609fe82f5a service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:39.878439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4ecce-4d67-48a7-8cac-be609fe82f5a service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:39.878733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4ecce-4d67-48a7-8cac-be609fe82f5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:39.880751 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75b4ecce-4d67-48a7-8cac-be609fe82f5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:40.002850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4ecce-4d67-48a7-8cac-be609fe82f5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:40.004612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4ecce-4d67-48a7-8cac-be609fe82f5a service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:40.029708 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 681/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:52:43.387123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c26c56-8556-4cc4-a052-15fd58f8253a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:43.388841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c26c56-8556-4cc4-a052-15fd58f8253a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:43.389297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c26c56-8556-4cc4-a052-15fd58f8253a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:43.391829 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 683/1364] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:43.401480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77224f40-e29f-449a-9f98-57cd1f657c33 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:43.402287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77224f40-e29f-449a-9f98-57cd1f657c33 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:43.402445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77224f40-e29f-449a-9f98-57cd1f657c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:43.423041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:43.500431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-77224f40-e29f-449a-9f98-57cd1f657c33 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:43.501871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-77224f40-e29f-449a-9f98-57cd1f657c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:43.653253 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 682/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:43.773140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd075391-eb3b-4f35-9cde-c49ec4f1927c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:43.773441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd075391-eb3b-4f35-9cde-c49ec4f1927c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:43.773945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd075391-eb3b-4f35-9cde-c49ec4f1927c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:43.775318 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 684/1366] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:52:43.781696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:43.781998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:43.782412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:43.902173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:43.904252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:44.023560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:44.069009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597abda-5302-4fc1-8db8-d8a6bcb75657 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:44.069509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597abda-5302-4fc1-8db8-d8a6bcb75657 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:44.070025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597abda-5302-4fc1-8db8-d8a6bcb75657 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:44.072857 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 685/1367] 10.222.0.26 () {54 vars in 961 bytes} [Sun Sep 25 00:52:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:44.082465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043427fe-8c16-4a72-89bf-c18f553d172c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:44.083102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043427fe-8c16-4a72-89bf-c18f553d172c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:44.083638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043427fe-8c16-4a72-89bf-c18f553d172c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:44.157802 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 683/1368] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:52:44.212055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-043427fe-8c16-4a72-89bf-c18f553d172c None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:44.214357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-043427fe-8c16-4a72-89bf-c18f553d172c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:44.449266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:44.491590 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 686/1369] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Sep 25 00:52:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:44.506216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:44.508218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:44.574385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56e41c-f9e0-4a1f-9c39-75ef66081d9c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:44.574835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56e41c-f9e0-4a1f-9c39-75ef66081d9c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:44.575653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56e41c-f9e0-4a1f-9c39-75ef66081d9c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:44.578088 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 687/1370] 10.222.0.26 () {54 vars in 963 bytes} [Sun Sep 25 00:52:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:44.589043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1983a7-e9f5-4f44-a628-1dca28986c2f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:44.589890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1983a7-e9f5-4f44-a628-1dca28986c2f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:44.590335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1983a7-e9f5-4f44-a628-1dca28986c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:44.713475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad2dc2-66bb-4539-a88a-7b9920f00082 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:44.719893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5f1983a7-e9f5-4f44-a628-1dca28986c2f None None] MFA Rules not processed for user `009181b0acc24632a5614482cdebb970`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:44.720964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5f1983a7-e9f5-4f44-a628-1dca28986c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:44.825730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:44.851819 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 688/1371] 10.222.0.26 () {58 vars in 1060 bytes} [Sun Sep 25 00:52:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:44.865465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1983a7-e9f5-4f44-a628-1dca28986c2f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:44.882857 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1983a7-e9f5-4f44-a628-1dca28986c2f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:44.936347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf5f86-d826-480a-8296-b8e7bed50aa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:44.937396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5f86-d826-480a-8296-b8e7bed50aa2 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:44.938390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5f86-d826-480a-8296-b8e7bed50aa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:44.938856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5f86-d826-480a-8296-b8e7bed50aa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:44.941763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fbdf5f86-d826-480a-8296-b8e7bed50aa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:45.045330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:45.048502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0306c-9170-477c-9bc8-72122e7c7ffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:45.050090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0306c-9170-477c-9bc8-72122e7c7ffc service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:45.051595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0306c-9170-477c-9bc8-72122e7c7ffc service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:45.052477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0306c-9170-477c-9bc8-72122e7c7ffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:45.057684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8aa0306c-9170-477c-9bc8-72122e7c7ffc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:45.096657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf5f86-d826-480a-8296-b8e7bed50aa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:45.098835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf5f86-d826-480a-8296-b8e7bed50aa2 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:45.127690 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 684/1372] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:52:45.286724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0306c-9170-477c-9bc8-72122e7c7ffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:45.288597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0306c-9170-477c-9bc8-72122e7c7ffc service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:45.323420 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 689/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:52:48.286207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf984ef-0b38-4305-aa97-b76d37ef3b59 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:48.286997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf984ef-0b38-4305-aa97-b76d37ef3b59 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:48.286997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf984ef-0b38-4305-aa97-b76d37ef3b59 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:48.288218 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 685/1374] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:52:48.295235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d553f-d324-40d5-a8d7-97a078eb110f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:48.295921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d553f-d324-40d5-a8d7-97a078eb110f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:48.296097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d553f-d324-40d5-a8d7-97a078eb110f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:48.402422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-786d553f-d324-40d5-a8d7-97a078eb110f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:48.404330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-786d553f-d324-40d5-a8d7-97a078eb110f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:48.493243 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:48.572117 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 690/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:52:48.889385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf984ef-0b38-4305-aa97-b76d37ef3b59 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:48.891568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-baf984ef-0b38-4305-aa97-b76d37ef3b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:48.987227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:49.085654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf984ef-0b38-4305-aa97-b76d37ef3b59 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:49.296135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf20678-9e3a-4a2f-bc40-7ba899b41d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:49.297112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf20678-9e3a-4a2f-bc40-7ba899b41d8a service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:49.297112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf20678-9e3a-4a2f-bc40-7ba899b41d8a service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:49.297381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf20678-9e3a-4a2f-bc40-7ba899b41d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:49.299648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6cf20678-9e3a-4a2f-bc40-7ba899b41d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:49.352905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:49.445564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf20678-9e3a-4a2f-bc40-7ba899b41d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:49.448623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf20678-9e3a-4a2f-bc40-7ba899b41d8a service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:49.483087 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 686/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:52:52.075308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b0b81-4482-43ff-aad3-85fac7843de2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:52.076082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b0b81-4482-43ff-aad3-85fac7843de2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:52.076082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b0b81-4482-43ff-aad3-85fac7843de2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:52.077853 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 691/1377] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:52.084024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38533b-f276-4754-b192-1af9bd31cc1c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:52.084583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38533b-f276-4754-b192-1af9bd31cc1c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:52.084583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38533b-f276-4754-b192-1af9bd31cc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:52.186716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7c38533b-f276-4754-b192-1af9bd31cc1c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:52.189223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7c38533b-f276-4754-b192-1af9bd31cc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:52.374172 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 687/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:52.688367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b0b81-4482-43ff-aad3-85fac7843de2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:52.692328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f45b0b81-4482-43ff-aad3-85fac7843de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:52.898567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b0b81-4482-43ff-aad3-85fac7843de2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:52.957492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:53.066955 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f8602-1f57-4ba3-b99b-a5cb5ba304a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:53.067766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8602-1f57-4ba3-b99b-a5cb5ba304a3 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:53.068052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8602-1f57-4ba3-b99b-a5cb5ba304a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:53.068260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8602-1f57-4ba3-b99b-a5cb5ba304a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:53.070061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5d8f8602-1f57-4ba3-b99b-a5cb5ba304a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:53.232185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f8602-1f57-4ba3-b99b-a5cb5ba304a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:53.236113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f8602-1f57-4ba3-b99b-a5cb5ba304a3 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:53.273326 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 692/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:52:56.321829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230345f-871f-4564-b965-5fd42e0f33f8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:56.322426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230345f-871f-4564-b965-5fd42e0f33f8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:56.322426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230345f-871f-4564-b965-5fd42e0f33f8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:56.323799 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 688/1380] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:52:56.330000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169727e-084f-45bd-a945-6d2e0a830079 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:56.330664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169727e-084f-45bd-a945-6d2e0a830079 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:56.330664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169727e-084f-45bd-a945-6d2e0a830079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:56.403256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:56.422995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c169727e-084f-45bd-a945-6d2e0a830079 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:52:56.425076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c169727e-084f-45bd-a945-6d2e0a830079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:56.561341 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 693/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:52:56.867841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230345f-871f-4564-b965-5fd42e0f33f8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:52:56.870006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6230345f-871f-4564-b965-5fd42e0f33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:56.875680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:57.123501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230345f-871f-4564-b965-5fd42e0f33f8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:52:57.226050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:52:57.305695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77daa58-7466-4e51-840a-c86ea91fc736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:52:57.308282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77daa58-7466-4e51-840a-c86ea91fc736 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:52:57.308282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77daa58-7466-4e51-840a-c86ea91fc736 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:52:57.308282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77daa58-7466-4e51-840a-c86ea91fc736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:52:57.311444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a77daa58-7466-4e51-840a-c86ea91fc736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:52:57.476578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77daa58-7466-4e51-840a-c86ea91fc736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:52:57.478194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77daa58-7466-4e51-840a-c86ea91fc736 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:52:57.503473 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 689/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:00.295687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d906b1-b895-4ad4-a199-35fe332707c5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:00.297902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d906b1-b895-4ad4-a199-35fe332707c5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:00.298376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d906b1-b895-4ad4-a199-35fe332707c5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:00.300890 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 694/1383] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:53:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:00.313447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540efef-a0e6-4977-8d0b-77dfc0acc7ba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:00.314241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540efef-a0e6-4977-8d0b-77dfc0acc7ba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:00.314402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540efef-a0e6-4977-8d0b-77dfc0acc7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:00.417333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9540efef-a0e6-4977-8d0b-77dfc0acc7ba None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:00.418654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9540efef-a0e6-4977-8d0b-77dfc0acc7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:00.473745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:00.564816 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 690/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:00.926404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d906b1-b895-4ad4-a199-35fe332707c5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:00.932975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-91d906b1-b895-4ad4-a199-35fe332707c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:01.137611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:01.177249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d906b1-b895-4ad4-a199-35fe332707c5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:01.346680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4777591-40ce-41dc-916b-c6d5bfa92be3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:01.348568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4777591-40ce-41dc-916b-c6d5bfa92be3 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:01.348980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4777591-40ce-41dc-916b-c6d5bfa92be3 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:01.349367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4777591-40ce-41dc-916b-c6d5bfa92be3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:01.351218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d4777591-40ce-41dc-916b-c6d5bfa92be3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:01.515418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:01.531116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4777591-40ce-41dc-916b-c6d5bfa92be3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:01.533776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4777591-40ce-41dc-916b-c6d5bfa92be3 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:01.563952 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 695/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:04.318890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b0883-ab84-469d-9fdc-9e886b88719e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:04.319697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b0883-ab84-469d-9fdc-9e886b88719e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:04.319889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b0883-ab84-469d-9fdc-9e886b88719e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:04.321671 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 691/1386] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:04.330781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f87c-4645-4304-9fe2-a3b32cd8ba96 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:04.331568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f87c-4645-4304-9fe2-a3b32cd8ba96 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:04.331982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6f87c-4645-4304-9fe2-a3b32cd8ba96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:04.452891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-49e6f87c-4645-4304-9fe2-a3b32cd8ba96 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:04.454215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-49e6f87c-4645-4304-9fe2-a3b32cd8ba96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:04.631793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755d3a8-3c0b-4783-8330-843e68664aa7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:04.632204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755d3a8-3c0b-4783-8330-843e68664aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:04.632738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755d3a8-3c0b-4783-8330-843e68664aa7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:04.635485 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 692/1387] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:53:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:53:04.641203 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 696/1388] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:04.645087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:04.645579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:04.646073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:04.729733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:04.730965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:04.876432 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 693/1389] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:05.193343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8022b-d136-4fb0-afed-5c959ddfb9f8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:05.193696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8022b-d136-4fb0-afed-5c959ddfb9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:05.193900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8022b-d136-4fb0-afed-5c959ddfb9f8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:05.194431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:05.195263 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 697/1390] 10.222.0.26 () {54 vars in 962 bytes} [Sun Sep 25 00:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:53:05.196169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:05.201185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d0bcd-6db4-4378-9158-cab534426b6c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:05.201459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d0bcd-6db4-4378-9158-cab534426b6c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:05.201729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d0bcd-6db4-4378-9158-cab534426b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:05.233100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:05.332681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8d6d0bcd-6db4-4378-9158-cab534426b6c None None] MFA Rules not processed for user `bddb3327380d43ef9b24277a3b308b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:05.334787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8d6d0bcd-6db4-4378-9158-cab534426b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:05.347279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:05.385907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f74c9-e507-4b12-8123-40cffc18165c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:05.455780 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 698/1391] 10.222.0.26 () {58 vars in 1059 bytes} [Sun Sep 25 00:53:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:53:05.469811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d0bcd-6db4-4378-9158-cab534426b6c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:05.478974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d0bcd-6db4-4378-9158-cab534426b6c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:05.573167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e34cb-bc4f-4595-8a3e-2f7b608b868b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:05.574118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e34cb-bc4f-4595-8a3e-2f7b608b868b service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:05.574401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e34cb-bc4f-4595-8a3e-2f7b608b868b service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:05.575286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e34cb-bc4f-4595-8a3e-2f7b608b868b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:05.578216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-329e34cb-bc4f-4595-8a3e-2f7b608b868b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:05.589031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41aa373-771c-47cc-b68a-78f2946e3b04 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:05.589858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aa373-771c-47cc-b68a-78f2946e3b04 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:05.590284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aa373-771c-47cc-b68a-78f2946e3b04 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:05.590704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aa373-771c-47cc-b68a-78f2946e3b04 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:05.594095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a41aa373-771c-47cc-b68a-78f2946e3b04 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:05.611315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:05.714697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:05.715334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e34cb-bc4f-4595-8a3e-2f7b608b868b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:05.717210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e34cb-bc4f-4595-8a3e-2f7b608b868b service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:05.747228 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 694/1392] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:05.774001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41aa373-771c-47cc-b68a-78f2946e3b04 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:05.775967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41aa373-771c-47cc-b68a-78f2946e3b04 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:05.804606 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 699/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:06.707846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2497-eddc-4c61-9ee6-821724a269ea None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:06.708720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2497-eddc-4c61-9ee6-821724a269ea None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:06.708720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd2497-eddc-4c61-9ee6-821724a269ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:06.816297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fedd2497-eddc-4c61-9ee6-821724a269ea None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:06.818210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fedd2497-eddc-4c61-9ee6-821724a269ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:06.979237 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 695/1394] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 00:53:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:07.044516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e792b0-bc28-41a3-a429-d237857623c1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:07.045134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e792b0-bc28-41a3-a429-d237857623c1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:07.045426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e792b0-bc28-41a3-a429-d237857623c1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:07.048824 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 700/1395] 10.222.0.26 () {54 vars in 943 bytes} [Sun Sep 25 00:53:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:07.057455 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9cb0-aff8-4fe0-bc4e-37111dabe439 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:07.058191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9cb0-aff8-4fe0-bc4e-37111dabe439 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:07.058705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d9cb0-aff8-4fe0-bc4e-37111dabe439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:07.071254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:07.193081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-258d9cb0-aff8-4fe0-bc4e-37111dabe439 None None] MFA Rules not processed for user `58594193ba6c41329a0bd1ca8205c69a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:07.195049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-258d9cb0-aff8-4fe0-bc4e-37111dabe439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:07.352769 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 696/1396] 10.222.0.26 () {58 vars in 1040 bytes} [Sun Sep 25 00:53:07 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:07.362706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e792b0-bc28-41a3-a429-d237857623c1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:07.364286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e9e792b0-bc28-41a3-a429-d237857623c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:07.599521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e792b0-bc28-41a3-a429-d237857623c1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:07.673963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:07.753778 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eb52a-1799-4121-afac-1e9f1c23de48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:07.754402 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb52a-1799-4121-afac-1e9f1c23de48 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:07.754987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb52a-1799-4121-afac-1e9f1c23de48 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:07.754987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb52a-1799-4121-afac-1e9f1c23de48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:07.756735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-667eb52a-1799-4121-afac-1e9f1c23de48 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:07.902838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667eb52a-1799-4121-afac-1e9f1c23de48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:07.904663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667eb52a-1799-4121-afac-1e9f1c23de48 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:07.914659 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 701/1397] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 00:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 00:53:08.356932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6b5b-41cf-4b12-84aa-fdada2013d5f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:08.357680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6b5b-41cf-4b12-84aa-fdada2013d5f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:08.357994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6b5b-41cf-4b12-84aa-fdada2013d5f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:08.361015 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 697/1398] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:53:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:08.370230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5f685-8516-48c5-8f0a-0f3656a7fc29 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:08.370951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5f685-8516-48c5-8f0a-0f3656a7fc29 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:08.371119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5f685-8516-48c5-8f0a-0f3656a7fc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:08.488504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6be5f685-8516-48c5-8f0a-0f3656a7fc29 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:08.489821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6be5f685-8516-48c5-8f0a-0f3656a7fc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:08.550404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:08.684227 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 702/1399] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:09.030945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb6b5b-41cf-4b12-84aa-fdada2013d5f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:09.034043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-daeb6b5b-41cf-4b12-84aa-fdada2013d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:09.137834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:09.287821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb6b5b-41cf-4b12-84aa-fdada2013d5f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:09.503616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a9e83-ddcb-4740-8361-d6e4a1068537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:09.504766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9e83-ddcb-4740-8361-d6e4a1068537 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:09.504766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9e83-ddcb-4740-8361-d6e4a1068537 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:09.505051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9e83-ddcb-4740-8361-d6e4a1068537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:09.506567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3b1a9e83-ddcb-4740-8361-d6e4a1068537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:09.571626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:09.682643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a9e83-ddcb-4740-8361-d6e4a1068537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:09.685669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a9e83-ddcb-4740-8361-d6e4a1068537 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:09.729626 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 698/1400] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:12.529843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903888d6-63f5-46f9-96c0-c2e08ef1cc6c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:12.531206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903888d6-63f5-46f9-96c0-c2e08ef1cc6c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:12.531206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903888d6-63f5-46f9-96c0-c2e08ef1cc6c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:12.536654 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 703/1401] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:53:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:12.543245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5b81c-44a7-49dc-9855-7d2424de62f6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:12.543933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5b81c-44a7-49dc-9855-7d2424de62f6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:12.544441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5b81c-44a7-49dc-9855-7d2424de62f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:12.652984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-65e5b81c-44a7-49dc-9855-7d2424de62f6 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:12.654590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-65e5b81c-44a7-49dc-9855-7d2424de62f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:12.806702 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 699/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:13.145490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903888d6-63f5-46f9-96c0-c2e08ef1cc6c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:13.147363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-903888d6-63f5-46f9-96c0-c2e08ef1cc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:13.384339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903888d6-63f5-46f9-96c0-c2e08ef1cc6c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:13.399779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:13.546722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce7eaa-9d0f-4140-a5ac-a06b219e3851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:13.547433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce7eaa-9d0f-4140-a5ac-a06b219e3851 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:13.547674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce7eaa-9d0f-4140-a5ac-a06b219e3851 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:13.547954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce7eaa-9d0f-4140-a5ac-a06b219e3851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:13.549983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a8ce7eaa-9d0f-4140-a5ac-a06b219e3851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:13.685909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce7eaa-9d0f-4140-a5ac-a06b219e3851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:13.689760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce7eaa-9d0f-4140-a5ac-a06b219e3851 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:13.736441 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 704/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:16.511228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e26d9-1418-464c-8f5f-b84bc329fec8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:16.512712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e26d9-1418-464c-8f5f-b84bc329fec8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:16.513210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e26d9-1418-464c-8f5f-b84bc329fec8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:16.515813 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 700/1404] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:53:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:16.525143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cef3b-1a8e-46e0-9660-fb31e3384c87 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:16.526029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cef3b-1a8e-46e0-9660-fb31e3384c87 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:16.526029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cef3b-1a8e-46e0-9660-fb31e3384c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:16.559556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:16.628103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-566cef3b-1a8e-46e0-9660-fb31e3384c87 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:16.629721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-566cef3b-1a8e-46e0-9660-fb31e3384c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:16.764535 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 705/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:17.161485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e26d9-1418-464c-8f5f-b84bc329fec8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:17.164344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-510e26d9-1418-464c-8f5f-b84bc329fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:17.173468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:17.407046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e26d9-1418-464c-8f5f-b84bc329fec8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:17.562310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:17.626919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa330d-a84b-42be-bd8d-6325789d7edc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:17.627769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa330d-a84b-42be-bd8d-6325789d7edc service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:17.628054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa330d-a84b-42be-bd8d-6325789d7edc service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:17.628329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa330d-a84b-42be-bd8d-6325789d7edc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:17.630024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-05fa330d-a84b-42be-bd8d-6325789d7edc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:17.766837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa330d-a84b-42be-bd8d-6325789d7edc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:17.768564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa330d-a84b-42be-bd8d-6325789d7edc service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:17.800254 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 701/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:20.378624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc6867-13bb-41f2-bc4a-a6c22de0e578 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:20.379439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc6867-13bb-41f2-bc4a-a6c22de0e578 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:20.379589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc6867-13bb-41f2-bc4a-a6c22de0e578 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:20.381002 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 706/1407] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:53:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:20.387611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b56003-a93d-4c7c-b678-cef93e8c6e38 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:20.388158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b56003-a93d-4c7c-b678-cef93e8c6e38 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:20.388158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b56003-a93d-4c7c-b678-cef93e8c6e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:20.477275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-46b56003-a93d-4c7c-b678-cef93e8c6e38 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:20.479627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-46b56003-a93d-4c7c-b678-cef93e8c6e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:20.551114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:20.654136 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 702/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:20.949547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc6867-13bb-41f2-bc4a-a6c22de0e578 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:20.954015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-10dc6867-13bb-41f2-bc4a-a6c22de0e578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:21.073527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:21.152576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc6867-13bb-41f2-bc4a-a6c22de0e578 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:21.309311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8aba7-dfab-44ca-b32d-08ba62e348a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:21.310060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8aba7-dfab-44ca-b32d-08ba62e348a8 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:21.310231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8aba7-dfab-44ca-b32d-08ba62e348a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:21.310461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8aba7-dfab-44ca-b32d-08ba62e348a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:21.315084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5dd8aba7-dfab-44ca-b32d-08ba62e348a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:21.397360 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:21.455747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8aba7-dfab-44ca-b32d-08ba62e348a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:21.457168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8aba7-dfab-44ca-b32d-08ba62e348a8 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:21.483519 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 707/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:30.652282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=25 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:31.477950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=20 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:33.442476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0e6a2-37d4-4682-bfeb-6aa20edd505f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:33.443047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0e6a2-37d4-4682-bfeb-6aa20edd505f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:33.443047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0e6a2-37d4-4682-bfeb-6aa20edd505f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:33.444487 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 703/1410] 10.222.0.26 () {54 vars in 918 bytes} [Sun Sep 25 00:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:33.450183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512c34-eba5-49e4-9043-34f2f9aa0895 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:33.450710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512c34-eba5-49e4-9043-34f2f9aa0895 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:33.450710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512c34-eba5-49e4-9043-34f2f9aa0895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:33.532715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b9512c34-eba5-49e4-9043-34f2f9aa0895 None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:33.533954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b9512c34-eba5-49e4-9043-34f2f9aa0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:33.663327 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 708/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Sun Sep 25 00:53:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:53:33.677787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dfa54-83d5-4887-a410-5d145c3ba7e5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:33.678077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dfa54-83d5-4887-a410-5d145c3ba7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:33.678352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dfa54-83d5-4887-a410-5d145c3ba7e5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:33.679711 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 704/1412] 10.222.0.26 () {54 vars in 918 bytes} [Sun Sep 25 00:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:33.684969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cefef-6eb4-4eaf-a946-fd393acbf7c8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:33.685256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cefef-6eb4-4eaf-a946-fd393acbf7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:33.686016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cefef-6eb4-4eaf-a946-fd393acbf7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:33.771613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:33.775352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-459cefef-6eb4-4eaf-a946-fd393acbf7c8 None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:33.777256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-459cefef-6eb4-4eaf-a946-fd393acbf7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:33.904411 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 709/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Sun Sep 25 00:53:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:36.608838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d904a8-53fb-4782-837a-7f78f71d2a84 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:36.609496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d904a8-53fb-4782-837a-7f78f71d2a84 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:36.609496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d904a8-53fb-4782-837a-7f78f71d2a84 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:36.610634 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:36.616443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ff8c5-6acb-4cf0-9418-1239636c545c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:36.616927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ff8c5-6acb-4cf0-9418-1239636c545c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:36.617114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ff8c5-6acb-4cf0-9418-1239636c545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:36.690354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3a9ff8c5-6acb-4cf0-9418-1239636c545c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:36.691891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3a9ff8c5-6acb-4cf0-9418-1239636c545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:36.751926 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:36.802935 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:37.096731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d904a8-53fb-4782-837a-7f78f71d2a84 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:37.098450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-50d904a8-53fb-4782-837a-7f78f71d2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:37.272768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d904a8-53fb-4782-837a-7f78f71d2a84 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:37.362026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:37.405363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b423c-6473-40f8-85d1-e7dec374b1f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:37.405985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b423c-6473-40f8-85d1-e7dec374b1f1 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:37.406268 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b423c-6473-40f8-85d1-e7dec374b1f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:37.406539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b423c-6473-40f8-85d1-e7dec374b1f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:37.408192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1d9b423c-6473-40f8-85d1-e7dec374b1f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:37.501423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b423c-6473-40f8-85d1-e7dec374b1f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:37.503038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b423c-6473-40f8-85d1-e7dec374b1f1 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:37.521342 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 706/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:38.710421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4bbd1-dc70-4d34-823e-0c9f2b98bac2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:38.711142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4bbd1-dc70-4d34-823e-0c9f2b98bac2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:38.711142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4bbd1-dc70-4d34-823e-0c9f2b98bac2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:38.712250 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:38.720148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb48ed-bbe5-4d79-a463-b95915ab9dba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:38.720780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb48ed-bbe5-4d79-a463-b95915ab9dba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:38.720780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb48ed-bbe5-4d79-a463-b95915ab9dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:38.807662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eebb48ed-bbe5-4d79-a463-b95915ab9dba None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:38.808943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eebb48ed-bbe5-4d79-a463-b95915ab9dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:38.840010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:38.926329 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:39.079718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4bbd1-dc70-4d34-823e-0c9f2b98bac2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:39.081371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-23b4bbd1-dc70-4d34-823e-0c9f2b98bac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:39.255846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4bbd1-dc70-4d34-823e-0c9f2b98bac2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:39.299359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:39.376441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba50c91-817d-4936-bd90-737c8361949f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:39.376964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50c91-817d-4936-bd90-737c8361949f service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:39.377234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50c91-817d-4936-bd90-737c8361949f service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:39.377502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50c91-817d-4936-bd90-737c8361949f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:39.379084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4ba50c91-817d-4936-bd90-737c8361949f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:39.475493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba50c91-817d-4936-bd90-737c8361949f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:39.477134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba50c91-817d-4936-bd90-737c8361949f service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:39.502805 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 712/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:40.710646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4770fe-5b01-467b-850a-f4878142ea9c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:40.711166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4770fe-5b01-467b-850a-f4878142ea9c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:40.711166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4770fe-5b01-467b-850a-f4878142ea9c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:40.712503 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:40.720341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167cf68-8d30-48d9-ad68-dffcacc87431 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:40.720898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167cf68-8d30-48d9-ad68-dffcacc87431 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:40.721020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167cf68-8d30-48d9-ad68-dffcacc87431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:40.785645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:40.809336 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9167cf68-8d30-48d9-ad68-dffcacc87431 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:40.810556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9167cf68-8d30-48d9-ad68-dffcacc87431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:40.921577 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:53:41.077444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4770fe-5b01-467b-850a-f4878142ea9c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:41.079239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7c4770fe-5b01-467b-850a-f4878142ea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:41.239815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4770fe-5b01-467b-850a-f4878142ea9c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:41.252732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:41.362969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ac6cb-7765-4720-99eb-4258c3b7498f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:41.363496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac6cb-7765-4720-99eb-4258c3b7498f service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:41.363775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac6cb-7765-4720-99eb-4258c3b7498f service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:41.364046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ac6cb-7765-4720-99eb-4258c3b7498f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:41.365695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0e3ac6cb-7765-4720-99eb-4258c3b7498f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:41.464269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ac6cb-7765-4720-99eb-4258c3b7498f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:41.465976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ac6cb-7765-4720-99eb-4258c3b7498f service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:41.486940 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 709/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:42.724606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b7273-8b47-4edd-b26d-b8ff932faeb0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:42.725220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b7273-8b47-4edd-b26d-b8ff932faeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:42.725220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b7273-8b47-4edd-b26d-b8ff932faeb0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:42.726527 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:42.733680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330af6d-c72e-4797-88bb-c864e3138e3e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:42.734224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330af6d-c72e-4797-88bb-c864e3138e3e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:42.734224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330af6d-c72e-4797-88bb-c864e3138e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:42.755873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:42.820735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a330af6d-c72e-4797-88bb-c864e3138e3e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:42.822043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a330af6d-c72e-4797-88bb-c864e3138e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:42.937396 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:43.090212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b7273-8b47-4edd-b26d-b8ff932faeb0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:43.091796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-886b7273-8b47-4edd-b26d-b8ff932faeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:43.209965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:43.248789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b7273-8b47-4edd-b26d-b8ff932faeb0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:43.379811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99fab6-1061-4df6-b5c5-385a62954419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:43.380614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99fab6-1061-4df6-b5c5-385a62954419 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:43.380741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99fab6-1061-4df6-b5c5-385a62954419 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:43.380987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99fab6-1061-4df6-b5c5-385a62954419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:43.382605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aa99fab6-1061-4df6-b5c5-385a62954419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:43.464734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:43.479382 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99fab6-1061-4df6-b5c5-385a62954419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:43.481028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99fab6-1061-4df6-b5c5-385a62954419 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:43.500945 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 715/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:44.642982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ef6e-3031-4a8d-9d2f-a2d819ecf2aa None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:44.643523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ef6e-3031-4a8d-9d2f-a2d819ecf2aa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:44.643523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3ef6e-3031-4a8d-9d2f-a2d819ecf2aa None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:44.645079 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:53:44.652797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803f551-d1b0-4a60-9eb3-9496d29b86dd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:44.653446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803f551-d1b0-4a60-9eb3-9496d29b86dd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:44.653575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803f551-d1b0-4a60-9eb3-9496d29b86dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:44.738835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2803f551-d1b0-4a60-9eb3-9496d29b86dd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:44.740063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2803f551-d1b0-4a60-9eb3-9496d29b86dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:44.855515 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:45.008348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3ef6e-3031-4a8d-9d2f-a2d819ecf2aa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:45.009840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e7e3ef6e-3031-4a8d-9d2f-a2d819ecf2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:45.106035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:45.177097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3ef6e-3031-4a8d-9d2f-a2d819ecf2aa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:45.295296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097abcc-720f-42a6-981a-325464c0c75c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:45.295972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097abcc-720f-42a6-981a-325464c0c75c service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:45.296227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097abcc-720f-42a6-981a-325464c0c75c service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:45.296504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097abcc-720f-42a6-981a-325464c0c75c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:45.298330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5097abcc-720f-42a6-981a-325464c0c75c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:45.362147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:45.415756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097abcc-720f-42a6-981a-325464c0c75c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:45.417449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097abcc-720f-42a6-981a-325464c0c75c service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:45.436788 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 712/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:46.564415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20bc4-0e42-463c-a2dd-9e5838c55f85 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:46.565009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20bc4-0e42-463c-a2dd-9e5838c55f85 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:46.565163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20bc4-0e42-463c-a2dd-9e5838c55f85 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:46.566501 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:53:46.573900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5394-f4a6-417b-9f24-57b830fdaa37 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:46.574519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5394-f4a6-417b-9f24-57b830fdaa37 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:46.574519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c5394-f4a6-417b-9f24-57b830fdaa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:46.664034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d25c5394-f4a6-417b-9f24-57b830fdaa37 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:46.665308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d25c5394-f4a6-417b-9f24-57b830fdaa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:46.786110 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:46.943759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd20bc4-0e42-463c-a2dd-9e5838c55f85 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:46.945331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4dd20bc4-0e42-463c-a2dd-9e5838c55f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:46.995535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:47.099175 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd20bc4-0e42-463c-a2dd-9e5838c55f85 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:47.223682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf6fa3-86ff-4e70-97c3-e49b8abc14c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:47.224309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf6fa3-86ff-4e70-97c3-e49b8abc14c8 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:47.224614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf6fa3-86ff-4e70-97c3-e49b8abc14c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:47.224893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf6fa3-86ff-4e70-97c3-e49b8abc14c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:47.226581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2daf6fa3-86ff-4e70-97c3-e49b8abc14c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:47.234121 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:47.324414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf6fa3-86ff-4e70-97c3-e49b8abc14c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:47.326626 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf6fa3-86ff-4e70-97c3-e49b8abc14c8 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:47.346638 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 718/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:48.439772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d89dc-cd70-4728-bc21-9b98a2b67eae None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:48.440412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d89dc-cd70-4728-bc21-9b98a2b67eae None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:48.440412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d89dc-cd70-4728-bc21-9b98a2b67eae None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:48.442553 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:48.452720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df1d79-2efb-4eb7-9291-682b42342500 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:48.453702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df1d79-2efb-4eb7-9291-682b42342500 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:48.453702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df1d79-2efb-4eb7-9291-682b42342500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:48.552024 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-14df1d79-2efb-4eb7-9291-682b42342500 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:48.553280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-14df1d79-2efb-4eb7-9291-682b42342500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:48.635522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:48.674262 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:53:48.883562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d89dc-cd70-4728-bc21-9b98a2b67eae None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:48.885128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3c6d89dc-cd70-4728-bc21-9b98a2b67eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:48.903425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:49.054848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d89dc-cd70-4728-bc21-9b98a2b67eae None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:49.156696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:49.186034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd493e-dd82-4ff4-91ac-249d2fa1fe0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:49.187067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd493e-dd82-4ff4-91ac-249d2fa1fe0d service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:49.187387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd493e-dd82-4ff4-91ac-249d2fa1fe0d service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:49.187818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd493e-dd82-4ff4-91ac-249d2fa1fe0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:49.190427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a1dd493e-dd82-4ff4-91ac-249d2fa1fe0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:49.332199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd493e-dd82-4ff4-91ac-249d2fa1fe0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:49.333926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd493e-dd82-4ff4-91ac-249d2fa1fe0d service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:49.354472 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 715/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:50.435233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fe717-1510-447f-8990-22a11bbb28af None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:50.435794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fe717-1510-447f-8990-22a11bbb28af None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:50.435794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fe717-1510-447f-8990-22a11bbb28af None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:50.437265 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:50.443795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4df7f4-08e2-4e4f-8962-f4f037e14f3c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:50.444340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4df7f4-08e2-4e4f-8962-f4f037e14f3c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:50.444340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4df7f4-08e2-4e4f-8962-f4f037e14f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:50.526720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bd4df7f4-08e2-4e4f-8962-f4f037e14f3c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:50.527972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bd4df7f4-08e2-4e4f-8962-f4f037e14f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:50.576360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:50.644056 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:50.795079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fe717-1510-447f-8990-22a11bbb28af None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:50.796645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a8fe717-1510-447f-8990-22a11bbb28af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:50.962993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fe717-1510-447f-8990-22a11bbb28af None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:51.032158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:51.098110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a99726-ba90-41e1-8a2e-efa52c946e63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:51.098709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a99726-ba90-41e1-8a2e-efa52c946e63 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:51.098986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a99726-ba90-41e1-8a2e-efa52c946e63 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:51.099277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a99726-ba90-41e1-8a2e-efa52c946e63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:51.100912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11a99726-ba90-41e1-8a2e-efa52c946e63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:51.197314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a99726-ba90-41e1-8a2e-efa52c946e63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:51.199056 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a99726-ba90-41e1-8a2e-efa52c946e63 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:51.220759 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 721/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:52.347837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761a35c-5453-4b3e-a3d8-a1dfcb0587c9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:52.348438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761a35c-5453-4b3e-a3d8-a1dfcb0587c9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:52.348438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761a35c-5453-4b3e-a3d8-a1dfcb0587c9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:52.350057 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:52.358967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa1ca4-031d-48e7-82a8-fa02749741fd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:52.359570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa1ca4-031d-48e7-82a8-fa02749741fd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:52.359570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa1ca4-031d-48e7-82a8-fa02749741fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:52.445158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-80aa1ca4-031d-48e7-82a8-fa02749741fd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:52.446390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-80aa1ca4-031d-48e7-82a8-fa02749741fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:52.448696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:52.558646 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:53:52.711465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761a35c-5453-4b3e-a3d8-a1dfcb0587c9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:52.713142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9761a35c-5453-4b3e-a3d8-a1dfcb0587c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:52.882701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761a35c-5453-4b3e-a3d8-a1dfcb0587c9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:52.913833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:53.012501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be641930-5dc6-4a28-90f6-6d6456fd5caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:53.013147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641930-5dc6-4a28-90f6-6d6456fd5caf service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:53.013429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641930-5dc6-4a28-90f6-6d6456fd5caf service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:53.013706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641930-5dc6-4a28-90f6-6d6456fd5caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:53.015471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-be641930-5dc6-4a28-90f6-6d6456fd5caf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:53.113399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be641930-5dc6-4a28-90f6-6d6456fd5caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:53.115068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be641930-5dc6-4a28-90f6-6d6456fd5caf service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:53.133569 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 718/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:53:55.302348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934ed8-4e9d-49d3-9460-225dd29b6776 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:55.303159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934ed8-4e9d-49d3-9460-225dd29b6776 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:55.303159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934ed8-4e9d-49d3-9460-225dd29b6776 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:55.304275 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:53:55.311610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57150603-8546-439b-93e7-4f4880026a37 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:55.311610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57150603-8546-439b-93e7-4f4880026a37 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:55.312107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57150603-8546-439b-93e7-4f4880026a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:55.361894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:55.395712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-57150603-8546-439b-93e7-4f4880026a37 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:55.397076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57150603-8546-439b-93e7-4f4880026a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:55.515907 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:53:55.810800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934ed8-4e9d-49d3-9460-225dd29b6776 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:55.813003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-50934ed8-4e9d-49d3-9460-225dd29b6776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:55.927133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934ed8-4e9d-49d3-9460-225dd29b6776 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:55.960989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:56.010454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ab458-e15f-4fdb-98d3-60ba8c2ae60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:56.011083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab458-e15f-4fdb-98d3-60ba8c2ae60a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:56.011359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab458-e15f-4fdb-98d3-60ba8c2ae60a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:56.011635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ab458-e15f-4fdb-98d3-60ba8c2ae60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:56.013820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff5ab458-e15f-4fdb-98d3-60ba8c2ae60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:56.145414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab458-e15f-4fdb-98d3-60ba8c2ae60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:56.146765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ab458-e15f-4fdb-98d3-60ba8c2ae60a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:56.166254 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 724/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:53:58.644961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74716-7434-4040-82dc-5c11a0db8914 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:58.645521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74716-7434-4040-82dc-5c11a0db8914 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:58.645521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74716-7434-4040-82dc-5c11a0db8914 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:58.646830 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:53:58.653068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca890a5-8efc-44f7-a822-5b347b569032 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:58.653647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca890a5-8efc-44f7-a822-5b347b569032 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:58.653758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca890a5-8efc-44f7-a822-5b347b569032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:58.733831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7ca890a5-8efc-44f7-a822-5b347b569032 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:53:58.735113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7ca890a5-8efc-44f7-a822-5b347b569032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:58.750318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:58.853927 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:53:59.142035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b74716-7434-4040-82dc-5c11a0db8914 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:53:59.143716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60b74716-7434-4040-82dc-5c11a0db8914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:59.262223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b74716-7434-4040-82dc-5c11a0db8914 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:53:59.276825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:59.344318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c1710-0068-431f-9a4c-a39b205fa668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:53:59.345046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c1710-0068-431f-9a4c-a39b205fa668 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:53:59.345320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c1710-0068-431f-9a4c-a39b205fa668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:53:59.345590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c1710-0068-431f-9a4c-a39b205fa668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:53:59.347243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-423c1710-0068-431f-9a4c-a39b205fa668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:53:59.480690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c1710-0068-431f-9a4c-a39b205fa668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:53:59.482925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c1710-0068-431f-9a4c-a39b205fa668 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:53:59.503369 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 721/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:02.223846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21212086-7449-4813-ae3d-beb387d6ca60 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:02.224455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21212086-7449-4813-ae3d-beb387d6ca60 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:02.224455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21212086-7449-4813-ae3d-beb387d6ca60 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:02.225782 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:02.231805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76acde6-65b1-4faf-93a8-4c1dbd308cf3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:02.232416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76acde6-65b1-4faf-93a8-4c1dbd308cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:02.232416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76acde6-65b1-4faf-93a8-4c1dbd308cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:02.287399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:02.308515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b76acde6-65b1-4faf-93a8-4c1dbd308cf3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:02.310253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b76acde6-65b1-4faf-93a8-4c1dbd308cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:02.533120 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:02.819827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21212086-7449-4813-ae3d-beb387d6ca60 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:02.822458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21212086-7449-4813-ae3d-beb387d6ca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:02.942205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21212086-7449-4813-ae3d-beb387d6ca60 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:02.989335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:03.026609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6f903-c2e3-4716-ba2b-b1d44ebc13ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:03.027221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f903-c2e3-4716-ba2b-b1d44ebc13ce service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:03.027502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f903-c2e3-4716-ba2b-b1d44ebc13ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:03.027773 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f903-c2e3-4716-ba2b-b1d44ebc13ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:03.032029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-38c6f903-c2e3-4716-ba2b-b1d44ebc13ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:03.199566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6f903-c2e3-4716-ba2b-b1d44ebc13ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:03.201579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6f903-c2e3-4716-ba2b-b1d44ebc13ce service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:03.221526 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 727/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:05.649295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a48dcf-817d-4312-b43c-2627ead5268f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:05.649812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a48dcf-817d-4312-b43c-2627ead5268f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:05.649812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a48dcf-817d-4312-b43c-2627ead5268f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:05.651142 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:05.658094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72953e72-f37b-4979-8a64-f35644dd49ea None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:05.658618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72953e72-f37b-4979-8a64-f35644dd49ea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:05.658714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72953e72-f37b-4979-8a64-f35644dd49ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:05.729468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-72953e72-f37b-4979-8a64-f35644dd49ea None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:05.730713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-72953e72-f37b-4979-8a64-f35644dd49ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:05.754545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:05.845685 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:06.136780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a48dcf-817d-4312-b43c-2627ead5268f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:06.139473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b8a48dcf-817d-4312-b43c-2627ead5268f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:06.268600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a48dcf-817d-4312-b43c-2627ead5268f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:06.291572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:06.360627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3565e10-b6dd-4fc4-a8e1-dd4e075050cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:06.361319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3565e10-b6dd-4fc4-a8e1-dd4e075050cc service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:06.361759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3565e10-b6dd-4fc4-a8e1-dd4e075050cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:06.362079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3565e10-b6dd-4fc4-a8e1-dd4e075050cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:06.364552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c3565e10-b6dd-4fc4-a8e1-dd4e075050cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:06.504620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3565e10-b6dd-4fc4-a8e1-dd4e075050cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:06.505997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3565e10-b6dd-4fc4-a8e1-dd4e075050cc service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:06.528254 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 724/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:54:08.911859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494262ac-90fa-4baa-b782-12ec539d76c8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:08.912523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494262ac-90fa-4baa-b782-12ec539d76c8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:08.912523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494262ac-90fa-4baa-b782-12ec539d76c8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:08.913843 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:08.920180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c038d3-4ee4-474e-b77b-134912960c2a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:08.920852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c038d3-4ee4-474e-b77b-134912960c2a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:08.920852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c038d3-4ee4-474e-b77b-134912960c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:09.012116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:09.016118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-38c038d3-4ee4-474e-b77b-134912960c2a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:09.017833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-38c038d3-4ee4-474e-b77b-134912960c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:09.147160 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:54:09.447084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494262ac-90fa-4baa-b782-12ec539d76c8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:09.448671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-494262ac-90fa-4baa-b782-12ec539d76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:09.583977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494262ac-90fa-4baa-b782-12ec539d76c8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:09.670576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:09.708695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38ef4c-9a74-4906-976d-f45888a8a937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:09.709222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ef4c-9a74-4906-976d-f45888a8a937 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:09.709504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ef4c-9a74-4906-976d-f45888a8a937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:09.709774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ef4c-9a74-4906-976d-f45888a8a937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:09.711470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd38ef4c-9a74-4906-976d-f45888a8a937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:09.875980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38ef4c-9a74-4906-976d-f45888a8a937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:09.878327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38ef4c-9a74-4906-976d-f45888a8a937 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:09.922573 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 730/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:12.369571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280f93e-bb00-4e12-86e7-3bba5da47be3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:12.370138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280f93e-bb00-4e12-86e7-3bba5da47be3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:12.370138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280f93e-bb00-4e12-86e7-3bba5da47be3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:12.371500 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:12.377405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eea07-2928-4ea8-bffd-2977d28b6cdc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:12.377947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eea07-2928-4ea8-bffd-2977d28b6cdc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:12.377947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eea07-2928-4ea8-bffd-2977d28b6cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:12.469976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c12eea07-2928-4ea8-bffd-2977d28b6cdc None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:12.471215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c12eea07-2928-4ea8-bffd-2977d28b6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:12.502880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:12.590633 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:12.883517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280f93e-bb00-4e12-86e7-3bba5da47be3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:12.885128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b280f93e-bb00-4e12-86e7-3bba5da47be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:12.997845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280f93e-bb00-4e12-86e7-3bba5da47be3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:13.042768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:13.094132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d9b5f-902e-48e6-9098-ea5ca1928e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:13.094755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9b5f-902e-48e6-9098-ea5ca1928e5d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:13.095030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9b5f-902e-48e6-9098-ea5ca1928e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:13.095300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9b5f-902e-48e6-9098-ea5ca1928e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:13.096977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dc2d9b5f-902e-48e6-9098-ea5ca1928e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:13.218445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d9b5f-902e-48e6-9098-ea5ca1928e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:13.219841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d9b5f-902e-48e6-9098-ea5ca1928e5d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:13.240903 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 727/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:15.601649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc6ea-8328-4e3e-a45f-6dd3b0f37259 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:15.602232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc6ea-8328-4e3e-a45f-6dd3b0f37259 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:15.602232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc6ea-8328-4e3e-a45f-6dd3b0f37259 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:15.603500 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:15.609961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286048c-f624-4551-b18c-55896cca1822 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:15.610560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286048c-f624-4551-b18c-55896cca1822 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:15.610560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286048c-f624-4551-b18c-55896cca1822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:15.684774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5286048c-f624-4551-b18c-55896cca1822 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:15.686000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5286048c-f624-4551-b18c-55896cca1822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:15.695170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:15.796586 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:16.086730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc6ea-8328-4e3e-a45f-6dd3b0f37259 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:16.089595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-619cc6ea-8328-4e3e-a45f-6dd3b0f37259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:16.214396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc6ea-8328-4e3e-a45f-6dd3b0f37259 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:16.278296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:16.298168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27d8e2-733e-4a11-be7c-436478a501fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:16.298788 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d8e2-733e-4a11-be7c-436478a501fc service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:16.299065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d8e2-733e-4a11-be7c-436478a501fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:16.299339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27d8e2-733e-4a11-be7c-436478a501fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:16.301020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a27d8e2-733e-4a11-be7c-436478a501fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:16.423297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27d8e2-733e-4a11-be7c-436478a501fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:16.424694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27d8e2-733e-4a11-be7c-436478a501fc service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:16.444336 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 733/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:18.750157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbaccc-e9f0-4838-b0d2-55e46a30bf98 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:18.750690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbaccc-e9f0-4838-b0d2-55e46a30bf98 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:18.750690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbaccc-e9f0-4838-b0d2-55e46a30bf98 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:18.751974 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:18.757904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd34f-92d5-4349-a953-cff8b2b8f8d6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:18.758426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd34f-92d5-4349-a953-cff8b2b8f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:18.758426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dd34f-92d5-4349-a953-cff8b2b8f8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:18.830816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c71dd34f-92d5-4349-a953-cff8b2b8f8d6 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:18.832071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c71dd34f-92d5-4349-a953-cff8b2b8f8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:18.874981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:18.943123 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:19.227807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbaccc-e9f0-4838-b0d2-55e46a30bf98 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:19.229577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c3fbaccc-e9f0-4838-b0d2-55e46a30bf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:19.345520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbaccc-e9f0-4838-b0d2-55e46a30bf98 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:19.390568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:19.432794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5205a9-fa5a-4087-a9c8-4ef5cdda4c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:19.433414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5205a9-fa5a-4087-a9c8-4ef5cdda4c09 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:19.433696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5205a9-fa5a-4087-a9c8-4ef5cdda4c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:19.434482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5205a9-fa5a-4087-a9c8-4ef5cdda4c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:19.436591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec5205a9-fa5a-4087-a9c8-4ef5cdda4c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:19.555335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5205a9-fa5a-4087-a9c8-4ef5cdda4c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:19.556837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5205a9-fa5a-4087-a9c8-4ef5cdda4c09 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:19.588438 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 730/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:21.897631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc57fc-da8d-4ca5-b397-670923246141 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:21.898299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc57fc-da8d-4ca5-b397-670923246141 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:21.898299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc57fc-da8d-4ca5-b397-670923246141 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:21.899560 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:21.905452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb22c7-f197-4a6b-88cf-6467a183b16f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:21.905936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb22c7-f197-4a6b-88cf-6467a183b16f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:21.906044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb22c7-f197-4a6b-88cf-6467a183b16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:21.977519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1abb22c7-f197-4a6b-88cf-6467a183b16f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:21.978763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1abb22c7-f197-4a6b-88cf-6467a183b16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:21.999919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:22.088573 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:54:22.389404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc57fc-da8d-4ca5-b397-670923246141 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:22.390969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-28fc57fc-da8d-4ca5-b397-670923246141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:22.523117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc57fc-da8d-4ca5-b397-670923246141 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:22.602779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:22.610340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74814e-31d7-4f96-a89a-e9fdef899be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:22.610954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74814e-31d7-4f96-a89a-e9fdef899be2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:22.611240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74814e-31d7-4f96-a89a-e9fdef899be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:22.611500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74814e-31d7-4f96-a89a-e9fdef899be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:22.613211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ce74814e-31d7-4f96-a89a-e9fdef899be2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:22.738341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74814e-31d7-4f96-a89a-e9fdef899be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:22.739851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74814e-31d7-4f96-a89a-e9fdef899be2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:22.761512 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 736/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:24.978642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d716-cd65-427c-b5bc-0ff52ff79f8b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:24.979224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d716-cd65-427c-b5bc-0ff52ff79f8b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:24.979224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d716-cd65-427c-b5bc-0ff52ff79f8b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:24.980955 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:54:24.987531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002e740-7929-44d8-840f-3fd0deffeb15 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:24.988500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002e740-7929-44d8-840f-3fd0deffeb15 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:24.988690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002e740-7929-44d8-840f-3fd0deffeb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:25.066191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c002e740-7929-44d8-840f-3fd0deffeb15 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:25.067427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c002e740-7929-44d8-840f-3fd0deffeb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:25.118197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:25.169696 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:54:25.456012 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445d716-cd65-427c-b5bc-0ff52ff79f8b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:25.457630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8445d716-cd65-427c-b5bc-0ff52ff79f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:25.566638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445d716-cd65-427c-b5bc-0ff52ff79f8b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:25.616157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:25.649749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a191977-eb38-4bcf-b503-df67aab0a4a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:25.650365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191977-eb38-4bcf-b503-df67aab0a4a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:25.650643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191977-eb38-4bcf-b503-df67aab0a4a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:25.651145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191977-eb38-4bcf-b503-df67aab0a4a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:25.653379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a191977-eb38-4bcf-b503-df67aab0a4a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:25.766936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a191977-eb38-4bcf-b503-df67aab0a4a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:25.768463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a191977-eb38-4bcf-b503-df67aab0a4a6 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:25.787448 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 733/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:35.168247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=22 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:35.785153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=66 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:46.672138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aaff-c6bf-4596-bec8-d9e6643a8e49 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:46.672889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aaff-c6bf-4596-bec8-d9e6643a8e49 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:46.672889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aaff-c6bf-4596-bec8-d9e6643a8e49 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:46.674167 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 738/1471] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:46.680246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482a51-966b-48b3-83c1-ca66f8f0feb7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:46.680861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482a51-966b-48b3-83c1-ca66f8f0feb7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:46.680861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95482a51-966b-48b3-83c1-ca66f8f0feb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:46.755760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-95482a51-966b-48b3-83c1-ca66f8f0feb7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:46.757193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95482a51-966b-48b3-83c1-ca66f8f0feb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:46.863912 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 734/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:47.166314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dc72b-8be7-4026-b2bf-b833a1352d02 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:47.166593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dc72b-8be7-4026-b2bf-b833a1352d02 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:47.166870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dc72b-8be7-4026-b2bf-b833a1352d02 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:47.168916 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 739/1473] 10.222.0.26 () {54 vars in 962 bytes} [Sun Sep 25 00:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:47.174287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0221e-237a-4bd5-9127-e06873db9b8f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:47.174569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0221e-237a-4bd5-9127-e06873db9b8f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:47.174842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0221e-237a-4bd5-9127-e06873db9b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:47.257825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fbf0221e-237a-4bd5-9127-e06873db9b8f None None] MFA Rules not processed for user `bddb3327380d43ef9b24277a3b308b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:47.259044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fbf0221e-237a-4bd5-9127-e06873db9b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:47.280983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:47.353777 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 735/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Sun Sep 25 00:54:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:47.363833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dc72b-8be7-4026-b2bf-b833a1352d02 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:47.365423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-620dc72b-8be7-4026-b2bf-b833a1352d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:47.480915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dc72b-8be7-4026-b2bf-b833a1352d02 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:47.562730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b054e54-5dc7-40bb-adb9-720def767208 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:47.563351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054e54-5dc7-40bb-adb9-720def767208 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:47.563623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054e54-5dc7-40bb-adb9-720def767208 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:47.563895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b054e54-5dc7-40bb-adb9-720def767208 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:47.565626 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0b054e54-5dc7-40bb-adb9-720def767208 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:47.630606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:47.697505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054e54-5dc7-40bb-adb9-720def767208 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:47.702544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b054e54-5dc7-40bb-adb9-720def767208 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:47.727131 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 740/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:54:50.068429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92245f88-7410-4dcf-9310-f41509deb38a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:50.069055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92245f88-7410-4dcf-9310-f41509deb38a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:50.069055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92245f88-7410-4dcf-9310-f41509deb38a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:50.070328 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 736/1476] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:50.077124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7acd9-3117-4d5c-810c-3197e0dada92 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:50.077646 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7acd9-3117-4d5c-810c-3197e0dada92 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:50.077768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7acd9-3117-4d5c-810c-3197e0dada92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:50.149376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-70e7acd9-3117-4d5c-810c-3197e0dada92 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:50.150653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70e7acd9-3117-4d5c-810c-3197e0dada92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:50.255636 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 741/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:50.531296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92245f88-7410-4dcf-9310-f41509deb38a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:50.532900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-92245f88-7410-4dcf-9310-f41509deb38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:50.672652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92245f88-7410-4dcf-9310-f41509deb38a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:50.731032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:50.782099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644a738-23e4-4f6e-aaf2-d7a8aa3431b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:50.782769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644a738-23e4-4f6e-aaf2-d7a8aa3431b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:50.783153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644a738-23e4-4f6e-aaf2-d7a8aa3431b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:50.783534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644a738-23e4-4f6e-aaf2-d7a8aa3431b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:50.784285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644a738-23e4-4f6e-aaf2-d7a8aa3431b1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:50.786195 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-b644a738-23e4-4f6e-aaf2-d7a8aa3431b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:54:50.786195 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 00:54:50.786195 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 00:54:50.787385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644a738-23e4-4f6e-aaf2-d7a8aa3431b1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:50.807790 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 737/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Sun Sep 25 00:54:50 2022] GET /identity/v3/projects => generated 2553 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 00:54:53.110112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688537ef-c35b-4e02-9caf-2a42d209fa9b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:53.110756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688537ef-c35b-4e02-9caf-2a42d209fa9b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:53.110756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688537ef-c35b-4e02-9caf-2a42d209fa9b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:53.112049 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 742/1479] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:53.117926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d328f-474c-4429-aa89-fb3b2b3cb71b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:53.118441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d328f-474c-4429-aa89-fb3b2b3cb71b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:53.118595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d328f-474c-4429-aa89-fb3b2b3cb71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:53.191362 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-217d328f-474c-4429-aa89-fb3b2b3cb71b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:53.192548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-217d328f-474c-4429-aa89-fb3b2b3cb71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:53.302719 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 738/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:53.602401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688537ef-c35b-4e02-9caf-2a42d209fa9b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:53.604352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-688537ef-c35b-4e02-9caf-2a42d209fa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:53.617594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:53.760340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688537ef-c35b-4e02-9caf-2a42d209fa9b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:53.859694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:53.907500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf002c-4549-45ad-afd9-7d611988d1bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:53.908341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf002c-4549-45ad-afd9-7d611988d1bb service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:53.908887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf002c-4549-45ad-afd9-7d611988d1bb service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:53.909376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf002c-4549-45ad-afd9-7d611988d1bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:53.912348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-30bf002c-4549-45ad-afd9-7d611988d1bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:54.038733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf002c-4549-45ad-afd9-7d611988d1bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:54.041846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf002c-4549-45ad-afd9-7d611988d1bb service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:54.067434 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 743/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:56.501115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7b33-008a-407a-93ad-7fa90f87c2ab None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:56.501563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7b33-008a-407a-93ad-7fa90f87c2ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:56.501563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7b33-008a-407a-93ad-7fa90f87c2ab None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:56.503546 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 739/1482] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:56.510362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f1bb-0b7a-4f42-be39-99a053feda39 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:56.510922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f1bb-0b7a-4f42-be39-99a053feda39 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:56.510922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f1bb-0b7a-4f42-be39-99a053feda39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:56.595869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a740f1bb-0b7a-4f42-be39-99a053feda39 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:56.597044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a740f1bb-0b7a-4f42-be39-99a053feda39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:56.654889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:56.713769 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 744/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:54:57.007393 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7b33-008a-407a-93ad-7fa90f87c2ab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:54:57.008979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-679e7b33-008a-407a-93ad-7fa90f87c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:57.034342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:57.120870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7b33-008a-407a-93ad-7fa90f87c2ab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:54:57.222602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5332a8-15ac-4b8d-901b-12daf741b0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:54:57.223127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5332a8-15ac-4b8d-901b-12daf741b0ec service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:57.223405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5332a8-15ac-4b8d-901b-12daf741b0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:57.223668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5332a8-15ac-4b8d-901b-12daf741b0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:57.225332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cd5332a8-15ac-4b8d-901b-12daf741b0ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:57.310689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:57.373779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5332a8-15ac-4b8d-901b-12daf741b0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:54:57.375354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5332a8-15ac-4b8d-901b-12daf741b0ec service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:54:57.396183 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 740/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:54:59.693708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833f29-a4ed-449b-b7d8-b0d4c36cad54 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:59.694225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833f29-a4ed-449b-b7d8-b0d4c36cad54 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:59.694225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833f29-a4ed-449b-b7d8-b0d4c36cad54 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:59.695563 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 745/1485] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:54:59.701820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c59de-dab0-4a20-a95f-f8eb8d83996f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:54:59.702310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c59de-dab0-4a20-a95f-f8eb8d83996f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:54:59.702424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c59de-dab0-4a20-a95f-f8eb8d83996f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:54:59.775202 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-097c59de-dab0-4a20-a95f-f8eb8d83996f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:54:59.776454 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-097c59de-dab0-4a20-a95f-f8eb8d83996f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:54:59.884030 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 741/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:55:00.175957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a833f29-a4ed-449b-b7d8-b0d4c36cad54 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:55:00.177537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9a833f29-a4ed-449b-b7d8-b0d4c36cad54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:00.287861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a833f29-a4ed-449b-b7d8-b0d4c36cad54 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:55:00.377354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:00.380565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c005f0d-3682-469c-b9fc-b73ff036f199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:55:00.381806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c005f0d-3682-469c-b9fc-b73ff036f199 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:55:00.382297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c005f0d-3682-469c-b9fc-b73ff036f199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:55:00.382686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c005f0d-3682-469c-b9fc-b73ff036f199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:55:00.385752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c005f0d-3682-469c-b9fc-b73ff036f199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:00.512048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c005f0d-3682-469c-b9fc-b73ff036f199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:55:00.513868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c005f0d-3682-469c-b9fc-b73ff036f199 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:55:00.534428 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 746/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:55:02.945436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc041d1-9532-4442-91e1-e95211b8a119 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:55:02.946019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc041d1-9532-4442-91e1-e95211b8a119 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:55:02.946118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc041d1-9532-4442-91e1-e95211b8a119 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:55:02.947544 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 742/1488] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:55:02.955736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdd3e-8d19-4400-8dda-6068da1f562f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:55:02.956528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdd3e-8d19-4400-8dda-6068da1f562f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:55:02.956697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdd3e-8d19-4400-8dda-6068da1f562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:55:03.046580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-77dcdd3e-8d19-4400-8dda-6068da1f562f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:55:03.047831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-77dcdd3e-8d19-4400-8dda-6068da1f562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:03.104643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:03.156999 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 747/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:55:03.457219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc041d1-9532-4442-91e1-e95211b8a119 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:55:03.458774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2fc041d1-9532-4442-91e1-e95211b8a119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:03.498146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:03.569580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc041d1-9532-4442-91e1-e95211b8a119 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:55:03.659357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b259fc5-5d80-4407-b3c4-a36bc8cc858e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:55:03.659981 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b259fc5-5d80-4407-b3c4-a36bc8cc858e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:55:03.660262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b259fc5-5d80-4407-b3c4-a36bc8cc858e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:55:03.660571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b259fc5-5d80-4407-b3c4-a36bc8cc858e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:55:03.662250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b259fc5-5d80-4407-b3c4-a36bc8cc858e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:03.752669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:03.791742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b259fc5-5d80-4407-b3c4-a36bc8cc858e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:55:03.793841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b259fc5-5d80-4407-b3c4-a36bc8cc858e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:55:03.817012 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 743/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:55:06.457122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef1e9-acdd-4059-8b8c-f078815d0e07 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:55:06.457767 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef1e9-acdd-4059-8b8c-f078815d0e07 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:55:06.457909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef1e9-acdd-4059-8b8c-f078815d0e07 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:55:06.460234 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 748/1491] 10.222.0.26 () {54 vars in 927 bytes} [Sun Sep 25 00:55:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:55:06.469070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959423b8-fc12-4911-ac8c-4dedef551501 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:55:06.469876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959423b8-fc12-4911-ac8c-4dedef551501 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:55:06.469876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959423b8-fc12-4911-ac8c-4dedef551501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:55:06.550776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-959423b8-fc12-4911-ac8c-4dedef551501 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:55:06.552172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-959423b8-fc12-4911-ac8c-4dedef551501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:06.680243 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 744/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Sun Sep 25 00:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:55:07.009900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef1e9-acdd-4059-8b8c-f078815d0e07 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:55:07.012014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a59ef1e9-acdd-4059-8b8c-f078815d0e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:07.130674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef1e9-acdd-4059-8b8c-f078815d0e07 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:55:07.226562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992db0f1-43f1-4eff-95ec-b42069bcef7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:55:07.227449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992db0f1-43f1-4eff-95ec-b42069bcef7e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:55:07.227782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992db0f1-43f1-4eff-95ec-b42069bcef7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:55:07.228028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992db0f1-43f1-4eff-95ec-b42069bcef7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:55:07.229759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-992db0f1-43f1-4eff-95ec-b42069bcef7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:55:07.239544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:07.363030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992db0f1-43f1-4eff-95ec-b42069bcef7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:55:07.364505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992db0f1-43f1-4eff-95ec-b42069bcef7e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:55:07.509301 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 749/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:55:16.676788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=82 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:17.505898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=50 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:24.505260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6d87-fef5-4aeb-92cd-5faa1422e52a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:24.505838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6d87-fef5-4aeb-92cd-5faa1422e52a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:24.505838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6d87-fef5-4aeb-92cd-5faa1422e52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:24.589219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6b6d6d87-fef5-4aeb-92cd-5faa1422e52a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:56:24.590736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6b6d6d87-fef5-4aeb-92cd-5faa1422e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:56:24.713468 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 745/1494] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:56:24.721541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb538765-691f-4de2-ae26-ab2a33205474 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:24.722091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb538765-691f-4de2-ae26-ab2a33205474 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:24.722091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb538765-691f-4de2-ae26-ab2a33205474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:24.742113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bb538765-691f-4de2-ae26-ab2a33205474 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:56:24.743317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bb538765-691f-4de2-ae26-ab2a33205474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:56:24.751750 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 750/1495] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:56:24.756511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d6d87-fef5-4aeb-92cd-5faa1422e52a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:24.767350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d6d87-fef5-4aeb-92cd-5faa1422e52a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:24.772239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:24.772885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:24.773159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:24.773428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:24.774078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 00:56:24.774370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:24.776245 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:24.776245 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 00:56:24.776245 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 00:56:24.777176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:24.791123 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 746/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 00:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:56:24.799924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb538765-691f-4de2-ae26-ab2a33205474 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:24.809165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb538765-691f-4de2-ae26-ab2a33205474 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:24.813744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:24.814346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:24.814624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:24.814893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:24.815527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:24.817280 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING py.warnings [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:24.817280 dv-cinder-master-3-smb devstack@keystone.service[81500]: warnings.warn(msg) Sep 25 00:56:24.817280 dv-cinder-master-3-smb devstack@keystone.service[81500]: Sep 25 00:56:24.818461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:24.859484 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 751/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 00:56:24 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:56:24.863865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:24.874728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe297ca-23d5-4690-a643-86a536c3a8f1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:24.930075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:24.937455 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:24.938068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:24.938347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:24.938618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:24.939252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:24.940302 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:24.940302 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 00:56:24.940302 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 00:56:24.941557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:24.965489 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 747/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 00:56:24 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 00:56:24.970603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:24.981518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44bb0e-a260-4bc9-9c6a-dc38abc02002 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:24.986115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:24.986706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:24.986982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:24.987251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:24.987889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:24.989007 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING py.warnings [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:24.989007 dv-cinder-master-3-smb devstack@keystone.service[81500]: warnings.warn(msg) Sep 25 00:56:24.989007 dv-cinder-master-3-smb devstack@keystone.service[81500]: Sep 25 00:56:24.989924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:25.003569 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 752/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 00:56:25.008281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:25.019087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adcb27-8292-4b90-b519-55e932b08f0f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:25.024050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c0671-9961-4d1f-94cb-d9da33216501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:25.024691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c0671-9961-4d1f-94cb-d9da33216501 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.025592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c0671-9961-4d1f-94cb-d9da33216501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.025868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c0671-9961-4d1f-94cb-d9da33216501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.026510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c0671-9961-4d1f-94cb-d9da33216501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:25.027527 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-ce7c0671-9961-4d1f-94cb-d9da33216501 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:25.027527 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 00:56:25.027527 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 00:56:25.028442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c0671-9961-4d1f-94cb-d9da33216501 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:25.042310 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 748/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 00:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 00:56:25.046478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:25.056986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3925ca7-879d-4409-8b0b-588db3ac9359 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:25.061669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68af754-3801-4a8a-b788-3864409cdf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:25.062355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68af754-3801-4a8a-b788-3864409cdf83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.062656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68af754-3801-4a8a-b788-3864409cdf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.063042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68af754-3801-4a8a-b788-3864409cdf83 admin admin] PATH_INFO: `/v3/projects/6c52d05e1f304c8588280bdb953ccbb5/users/05e3c26a9f90429e8bc6d8ae307432c3/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.082714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68af754-3801-4a8a-b788-3864409cdf83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c52d05e1f304c8588280bdb953ccbb5, user_id=05e3c26a9f90429e8bc6d8ae307432c3, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:25.083732 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING py.warnings [None req-e68af754-3801-4a8a-b788-3864409cdf83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:25.083732 dv-cinder-master-3-smb devstack@keystone.service[81500]: warnings.warn(msg) Sep 25 00:56:25.083732 dv-cinder-master-3-smb devstack@keystone.service[81500]: Sep 25 00:56:25.085685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68af754-3801-4a8a-b788-3864409cdf83 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:25.098943 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 753/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 00:56:25 2022] PUT /identity/v3/projects/6c52d05e1f304c8588280bdb953ccbb5/users/05e3c26a9f90429e8bc6d8ae307432c3/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 00:56:25.111932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c97dee-800c-4aca-829b-28417a2eafc0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.112199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c97dee-800c-4aca-829b-28417a2eafc0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.112576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c97dee-800c-4aca-829b-28417a2eafc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.142485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-08c97dee-800c-4aca-829b-28417a2eafc0 None None] MFA Rules not processed for user `05e3c26a9f90429e8bc6d8ae307432c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:56:25.143918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08c97dee-800c-4aca-829b-28417a2eafc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:56:25.210847 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 749/1502] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:56:25.227684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dfa53-77bd-40d1-b767-8e64dcf7e093 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.227968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dfa53-77bd-40d1-b767-8e64dcf7e093 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.228517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213dfa53-77bd-40d1-b767-8e64dcf7e093 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.230438 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 754/1503] 10.222.0.26 () {54 vars in 963 bytes} [Sun Sep 25 00:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:56:25.237043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de251-6be9-4a6a-b2df-9cb7576a9963 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.237643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de251-6be9-4a6a-b2df-9cb7576a9963 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.237926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de251-6be9-4a6a-b2df-9cb7576a9963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.302196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c37de251-6be9-4a6a-b2df-9cb7576a9963 None None] MFA Rules not processed for user `009181b0acc24632a5614482cdebb970`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:56:25.303418 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c37de251-6be9-4a6a-b2df-9cb7576a9963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:56:25.391039 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 750/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Sun Sep 25 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:56:25.407225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dfa53-77bd-40d1-b767-8e64dcf7e093 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:25.419870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213dfa53-77bd-40d1-b767-8e64dcf7e093 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:25.423550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:25.424197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.424535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.424833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.433368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:25.434786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:25.443866 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 755/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:56:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:56:25.491966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ae57e-1728-455a-90a8-84772ed521dc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.492263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ae57e-1728-455a-90a8-84772ed521dc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.492582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ae57e-1728-455a-90a8-84772ed521dc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.493885 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 751/1506] 10.222.0.26 () {52 vars in 830 bytes} [Sun Sep 25 00:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:56:25.498266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:25.560106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3924e4-6fd3-4cfd-b47b-f57f7ce27dfb service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:25.564794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:25.565416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.565696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.565965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] PATH_INFO: `/v3/users/05e3c26a9f90429e8bc6d8ae307432c3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.567088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e3c26a9f90429e8bc6d8ae307432c3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:25.568096 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING py.warnings [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:25.568096 dv-cinder-master-3-smb devstack@keystone.service[81500]: warnings.warn(msg) Sep 25 00:56:25.568096 dv-cinder-master-3-smb devstack@keystone.service[81500]: Sep 25 00:56:25.569335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:25.639977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e3c26a9f90429e8bc6d8ae307432c3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:25.640655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e3c26a9f90429e8bc6d8ae307432c3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:25.655414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e3c26a9f90429e8bc6d8ae307432c3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:25.682092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e3c26a9f90429e8bc6d8ae307432c3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:25.682935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e3c26a9f90429e8bc6d8ae307432c3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:25.687612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:25.701613 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 756/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 00:56:25 2022] DELETE /identity/v3/users/05e3c26a9f90429e8bc6d8ae307432c3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:56:25.706035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.706514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.706650 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.726928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:56:25.728192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:56:25.741685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:25.797497 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 752/1508] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:56:25.807872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:25.809412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:56:25.891670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c601d-f5c3-46ab-aa60-c85bbfa854e6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:25.895355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1e97e-88af-4695-a750-ac6aca340bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:25.895956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e97e-88af-4695-a750-ac6aca340bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:25.896238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e97e-88af-4695-a750-ac6aca340bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:25.896810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e97e-88af-4695-a750-ac6aca340bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:25.914596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1e97e-88af-4695-a750-ac6aca340bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:25.916522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1e97e-88af-4695-a750-ac6aca340bb7 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:25.934774 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 757/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:56:26.346042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:56:26.347552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:56:26.361159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec7df5-6e0c-4034-b54c-625cee76e903 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:56:26.366844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:56:26.367524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:56:26.367807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:56:26.368085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] PATH_INFO: `/v3/projects/6c52d05e1f304c8588280bdb953ccbb5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:56:26.369261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c52d05e1f304c8588280bdb953ccbb5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:56:26.370315 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 00:56:26.370315 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 00:56:26.370315 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 00:56:26.371803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:56:26.436314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c52d05e1f304c8588280bdb953ccbb5', 'request_id': 'req-e9771847-31dc-4034-850f-cdf52140cc2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:26.450185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c52d05e1f304c8588280bdb953ccbb5', 'request_id': 'req-e9771847-31dc-4034-850f-cdf52140cc2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:26.456355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c52d05e1f304c8588280bdb953ccbb5', 'request_id': 'req-e9771847-31dc-4034-850f-cdf52140cc2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:26.473699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] The token cache is being invalidate because project 6c52d05e1f304c8588280bdb953ccbb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 00:56:26.474106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:26.475051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9771847-31dc-4034-850f-cdf52140cc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 00:56:26.477024 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 753/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 00:56:26 2022] DELETE /identity/v3/projects/6c52d05e1f304c8588280bdb953ccbb5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 00:56:35.928979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:36.469678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:05.493957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57242410-96ff-4945-9648-05bd6aff0552 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:05.494555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57242410-96ff-4945-9648-05bd6aff0552 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:05.494555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57242410-96ff-4945-9648-05bd6aff0552 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:05.495856 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:05.501102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163f58b-6aac-4ea5-86ee-747c511c2908 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:05.501559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163f58b-6aac-4ea5-86ee-747c511c2908 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:05.501665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163f58b-6aac-4ea5-86ee-747c511c2908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:05.521671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6163f58b-6aac-4ea5-86ee-747c511c2908 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:05.522892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6163f58b-6aac-4ea5-86ee-747c511c2908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:05.583301 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:05.744497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57242410-96ff-4945-9648-05bd6aff0552 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:05.746311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-57242410-96ff-4945-9648-05bd6aff0552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:05.859509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57242410-96ff-4945-9648-05bd6aff0552 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:05.864968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b3cac-1157-4246-8845-3bc2de69f348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:05.865574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b3cac-1157-4246-8845-3bc2de69f348 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:05.865882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b3cac-1157-4246-8845-3bc2de69f348 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:05.866151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b3cac-1157-4246-8845-3bc2de69f348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:05.874928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b3cac-1157-4246-8845-3bc2de69f348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:05.876553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b3cac-1157-4246-8845-3bc2de69f348 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:05.887052 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 759/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:58:07.041344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5deb3a-7c88-4a79-8374-0f3923f66dab None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:07.041906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5deb3a-7c88-4a79-8374-0f3923f66dab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:07.041906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5deb3a-7c88-4a79-8374-0f3923f66dab None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:07.043289 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:07.049777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8ad7-551c-464c-8520-0452fd15913e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:07.050246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8ad7-551c-464c-8520-0452fd15913e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:07.050361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8ad7-551c-464c-8520-0452fd15913e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:07.070398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dcec8ad7-551c-464c-8520-0452fd15913e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:07.072218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dcec8ad7-551c-464c-8520-0452fd15913e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:07.079899 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:07.235839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5deb3a-7c88-4a79-8374-0f3923f66dab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:07.250720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5deb3a-7c88-4a79-8374-0f3923f66dab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:07.256042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9885e-9245-46a1-acee-72744e678d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:07.256739 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9885e-9245-46a1-acee-72744e678d4d service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:07.257010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9885e-9245-46a1-acee-72744e678d4d service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:07.257279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9885e-9245-46a1-acee-72744e678d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:07.265395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9885e-9245-46a1-acee-72744e678d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:07.266999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9885e-9245-46a1-acee-72744e678d4d service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:07.276892 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 756/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:58:07.469549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db6d5-6dda-4855-be55-d99348be3d95 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:07.469835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db6d5-6dda-4855-be55-d99348be3d95 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:07.470113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440db6d5-6dda-4855-be55-d99348be3d95 None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:07.471415 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 761/1517] 10.222.0.25 () {54 vars in 925 bytes} [Sun Sep 25 00:58:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 00:58:07.480906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65776ed-7a6c-42be-a9df-6cb0c5e645b8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:07.481272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65776ed-7a6c-42be-a9df-6cb0c5e645b8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:07.481579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65776ed-7a6c-42be-a9df-6cb0c5e645b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:07.541856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f65776ed-7a6c-42be-a9df-6cb0c5e645b8 None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:07.543081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f65776ed-7a6c-42be-a9df-6cb0c5e645b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:07.623975 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 757/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Sun Sep 25 00:58:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:58:16.560513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e548ee-bc80-4ef2-b17f-543d67a240c8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:16.561082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e548ee-bc80-4ef2-b17f-543d67a240c8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:16.561082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e548ee-bc80-4ef2-b17f-543d67a240c8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:16.562436 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 762/1519] 10.222.0.26 () {54 vars in 961 bytes} [Sun Sep 25 00:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:16.569398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668aa0f-22a8-42cd-91f6-20f1dc4685ed None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:16.569976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668aa0f-22a8-42cd-91f6-20f1dc4685ed None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:16.569976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668aa0f-22a8-42cd-91f6-20f1dc4685ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:16.597267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6668aa0f-22a8-42cd-91f6-20f1dc4685ed None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:16.598665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6668aa0f-22a8-42cd-91f6-20f1dc4685ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:16.607139 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 758/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Sep 25 00:58:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 00:58:16.615312 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e548ee-bc80-4ef2-b17f-543d67a240c8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:16.629371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e548ee-bc80-4ef2-b17f-543d67a240c8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:16.634828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30576e1f-72f5-4b9d-9a01-def19fb91e5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:16.635424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30576e1f-72f5-4b9d-9a01-def19fb91e5d service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:16.635695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30576e1f-72f5-4b9d-9a01-def19fb91e5d service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:16.635984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30576e1f-72f5-4b9d-9a01-def19fb91e5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:16.644667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30576e1f-72f5-4b9d-9a01-def19fb91e5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:16.646246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30576e1f-72f5-4b9d-9a01-def19fb91e5d service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:16.658923 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 763/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:58:18.867285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d67195-79a4-4a9b-86d3-6546129873de None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:18.867873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d67195-79a4-4a9b-86d3-6546129873de None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:18.867873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d67195-79a4-4a9b-86d3-6546129873de None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:18.869257 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:18.885080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d09f6-385e-42a2-8668-65e9413269fb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:18.885592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d09f6-385e-42a2-8668-65e9413269fb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:18.885700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d09f6-385e-42a2-8668-65e9413269fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:18.916807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cb0d09f6-385e-42a2-8668-65e9413269fb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:18.918145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cb0d09f6-385e-42a2-8668-65e9413269fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:18.926930 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:19.091334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d67195-79a4-4a9b-86d3-6546129873de None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:19.106700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d67195-79a4-4a9b-86d3-6546129873de None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:19.112443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844852-b9fa-4133-b27c-62f31bf65a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:19.113065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844852-b9fa-4133-b27c-62f31bf65a38 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:19.113349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844852-b9fa-4133-b27c-62f31bf65a38 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:19.113623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844852-b9fa-4133-b27c-62f31bf65a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:19.123182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36844852-b9fa-4133-b27c-62f31bf65a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:19.124777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36844852-b9fa-4133-b27c-62f31bf65a38 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:19.135516 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 760/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:58:20.272172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466f0a-566f-4a84-b295-24807365c859 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:20.272919 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466f0a-566f-4a84-b295-24807365c859 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:20.272919 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466f0a-566f-4a84-b295-24807365c859 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:20.275074 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 25 00:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:20.283023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61c5a5-a6aa-4a16-b649-54a0d30b8c84 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:20.283593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61c5a5-a6aa-4a16-b649-54a0d30b8c84 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:20.283593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61c5a5-a6aa-4a16-b649-54a0d30b8c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:20.307881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6f61c5a5-a6aa-4a16-b649-54a0d30b8c84 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:20.309441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6f61c5a5-a6aa-4a16-b649-54a0d30b8c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:20.317294 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 25 00:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:20.470325 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466f0a-566f-4a84-b295-24807365c859 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:20.485256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466f0a-566f-4a84-b295-24807365c859 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:20.490743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a982eed-287a-4311-948c-bbc18f58b087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:20.491369 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a982eed-287a-4311-948c-bbc18f58b087 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:20.491608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a982eed-287a-4311-948c-bbc18f58b087 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:20.491900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a982eed-287a-4311-948c-bbc18f58b087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:20.500949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a982eed-287a-4311-948c-bbc18f58b087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:20.502519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a982eed-287a-4311-948c-bbc18f58b087 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:20.512600 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 766/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:58:24.737504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dcb0e-0684-472d-b302-5519bb02d558 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:24.738059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dcb0e-0684-472d-b302-5519bb02d558 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:24.738059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dcb0e-0684-472d-b302-5519bb02d558 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:24.739324 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:24.745246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0799b-aa72-42e5-8dc6-d9094b0209ab None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:24.745741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0799b-aa72-42e5-8dc6-d9094b0209ab None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:24.745849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0799b-aa72-42e5-8dc6-d9094b0209ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:24.766044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-27b0799b-aa72-42e5-8dc6-d9094b0209ab None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:24.770278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-27b0799b-aa72-42e5-8dc6-d9094b0209ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:24.776264 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:25.074266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dcb0e-0684-472d-b302-5519bb02d558 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:25.075995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2e8dcb0e-0684-472d-b302-5519bb02d558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:25.109462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:25.141518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dcb0e-0684-472d-b302-5519bb02d558 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:25.145293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f71d89-6f5c-4ea5-875a-59fa13a93622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:25.145835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f71d89-6f5c-4ea5-875a-59fa13a93622 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:25.146135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f71d89-6f5c-4ea5-875a-59fa13a93622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:25.146400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f71d89-6f5c-4ea5-875a-59fa13a93622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:25.157109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f71d89-6f5c-4ea5-875a-59fa13a93622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:25.158081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f71d89-6f5c-4ea5-875a-59fa13a93622 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:25.168491 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 763/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:58:27.361218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ef8b4-5cfa-4e8e-80d2-ea01dbefea7f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:27.361867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ef8b4-5cfa-4e8e-80d2-ea01dbefea7f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:27.361867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ef8b4-5cfa-4e8e-80d2-ea01dbefea7f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:27.363195 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:58:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:27.370005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa881-7dfc-4dcd-a2f9-e52aac0070a2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:27.370681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa881-7dfc-4dcd-a2f9-e52aac0070a2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:27.370681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa881-7dfc-4dcd-a2f9-e52aac0070a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:27.394516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-326aa881-7dfc-4dcd-a2f9-e52aac0070a2 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:27.396339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-326aa881-7dfc-4dcd-a2f9-e52aac0070a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:27.407857 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:27.731520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ef8b4-5cfa-4e8e-80d2-ea01dbefea7f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:27.739640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=68 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:27.747914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ef8b4-5cfa-4e8e-80d2-ea01dbefea7f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:27.751629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f5769-a510-473a-b8c8-2848fcadffa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:27.752305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f5769-a510-473a-b8c8-2848fcadffa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:27.752610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f5769-a510-473a-b8c8-2848fcadffa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:27.752883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f5769-a510-473a-b8c8-2848fcadffa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:27.763676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f5769-a510-473a-b8c8-2848fcadffa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:27.765348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f5769-a510-473a-b8c8-2848fcadffa7 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:27.777884 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 769/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:58:31.506787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b679d-f240-43d3-b118-634986447472 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:31.507394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b679d-f240-43d3-b118-634986447472 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:31.507394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b679d-f240-43d3-b118-634986447472 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:31.509334 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:58:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 00:58:31.515024 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741b092-2fe4-4b8b-97b8-8a6fa54f93c7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:31.516275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741b092-2fe4-4b8b-97b8-8a6fa54f93c7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:31.516275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741b092-2fe4-4b8b-97b8-8a6fa54f93c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:31.535987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c741b092-2fe4-4b8b-97b8-8a6fa54f93c7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:31.537318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c741b092-2fe4-4b8b-97b8-8a6fa54f93c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:31.546881 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:31.854245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b679d-f240-43d3-b118-634986447472 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:31.857564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-056b679d-f240-43d3-b118-634986447472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:31.922371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b679d-f240-43d3-b118-634986447472 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:31.926266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831e16a-8cc5-4c7f-a8ac-d268b33ed198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:31.927059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831e16a-8cc5-4c7f-a8ac-d268b33ed198 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:31.927538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831e16a-8cc5-4c7f-a8ac-d268b33ed198 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:31.927946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831e16a-8cc5-4c7f-a8ac-d268b33ed198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:31.938847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831e16a-8cc5-4c7f-a8ac-d268b33ed198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:31.940321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831e16a-8cc5-4c7f-a8ac-d268b33ed198 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:31.951270 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 766/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 00:58:34.273860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506f99-4a29-49ed-a0a3-c9f717658832 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:34.274530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506f99-4a29-49ed-a0a3-c9f717658832 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:34.274530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506f99-4a29-49ed-a0a3-c9f717658832 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:34.276039 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 25 00:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 00:58:34.281878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370b831-b9ba-4559-9ce6-63e108d1028f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:34.282382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370b831-b9ba-4559-9ce6-63e108d1028f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:34.282499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370b831-b9ba-4559-9ce6-63e108d1028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:34.310791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2370b831-b9ba-4559-9ce6-63e108d1028f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 00:58:34.312442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2370b831-b9ba-4559-9ce6-63e108d1028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:34.321446 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 25 00:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 00:58:34.647001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d506f99-4a29-49ed-a0a3-c9f717658832 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 00:58:34.649253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5d506f99-4a29-49ed-a0a3-c9f717658832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 00:58:34.666276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d506f99-4a29-49ed-a0a3-c9f717658832 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 00:58:34.670676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a3761-0a92-42f4-be02-e2f8091399a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 00:58:34.671641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a3761-0a92-42f4-be02-e2f8091399a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 00:58:34.672096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a3761-0a92-42f4-be02-e2f8091399a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 00:58:34.672607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a3761-0a92-42f4-be02-e2f8091399a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 00:58:34.687581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a3761-0a92-42f4-be02-e2f8091399a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 00:58:34.690476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a3761-0a92-42f4-be02-e2f8091399a0 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 00:58:34.708804 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 772/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 00:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 00:58:44.299278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=47 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:44.700102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=14 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:23.862706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a3490-f645-401a-ae49-fb9fe7ac92b1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:23.871056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1008a-83c2-46b2-a320-50577c04d50b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:23.871056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1008a-83c2-46b2-a320-50577c04d50b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:23.871056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1008a-83c2-46b2-a320-50577c04d50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:23.871768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a3490-f645-401a-ae49-fb9fe7ac92b1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:23.871768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a3490-f645-401a-ae49-fb9fe7ac92b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:23.907233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-040a3490-f645-401a-ae49-fb9fe7ac92b1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:23.910542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-040a3490-f645-401a-ae49-fb9fe7ac92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:23.916460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cfb1008a-83c2-46b2-a320-50577c04d50b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:23.917470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cfb1008a-83c2-46b2-a320-50577c04d50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:23.936000 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 773/1540] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:23.947649 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 768/1541] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:23.948618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4a0a4-4c1b-4b65-a9bc-c24bdfe51ccb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:23.949182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4a0a4-4c1b-4b65-a9bc-c24bdfe51ccb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:23.949737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4a0a4-4c1b-4b65-a9bc-c24bdfe51ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:23.958646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a2aa0-ccb6-499b-bdd5-f7f9f5dc0882 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:23.959126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a2aa0-ccb6-499b-bdd5-f7f9f5dc0882 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:23.959620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a2aa0-ccb6-499b-bdd5-f7f9f5dc0882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:23.998142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cdc4a0a4-4c1b-4b65-a9bc-c24bdfe51ccb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.006003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cdc4a0a4-4c1b-4b65-a9bc-c24bdfe51ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.014309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c15a2aa0-ccb6-499b-bdd5-f7f9f5dc0882 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.016841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c15a2aa0-ccb6-499b-bdd5-f7f9f5dc0882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.027029 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 774/1542] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:24.034219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4a0a4-4c1b-4b65-a9bc-c24bdfe51ccb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.047166 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 769/1543] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:24.049712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4a0a4-4c1b-4b65-a9bc-c24bdfe51ccb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.055567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a2aa0-ccb6-499b-bdd5-f7f9f5dc0882 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.065413 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.066914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.067482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.067988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.069308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:24.070055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.072687 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING py.warnings [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 25 01:00:24.072687 dv-cinder-master-3-smb devstack@keystone.service[81500]: warnings.warn(msg) Sep 25 01:00:24.072687 dv-cinder-master-3-smb devstack@keystone.service[81500]: Sep 25 01:00:24.074805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.084414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a2aa0-ccb6-499b-bdd5-f7f9f5dc0882 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.116169 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 775/1544] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:00:24.137633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.138904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.139387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.139551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.140015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.145417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:24.146004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.154247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.158985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8ef20-954d-465f-98a7-bd999ab9014f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.174521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.175711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.176067 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.176531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.177259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.179418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.192582 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 770/1545] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:00:24.202684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.240595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1544ed-a670-4740-b47c-324093c992dc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.241496 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 776/1546] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:24.247296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.416345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cdbe3-c31e-464d-b9bc-78ad50027039 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.430388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16075350-9e89-49c4-af3f-440dcda33edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.431256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16075350-9e89-49c4-af3f-440dcda33edf admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.431545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16075350-9e89-49c4-af3f-440dcda33edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.431815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16075350-9e89-49c4-af3f-440dcda33edf admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.432518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16075350-9e89-49c4-af3f-440dcda33edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.433573 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING py.warnings [None req-16075350-9e89-49c4-af3f-440dcda33edf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 01:00:24.433573 dv-cinder-master-3-smb devstack@keystone.service[81500]: warnings.warn(msg) Sep 25 01:00:24.433573 dv-cinder-master-3-smb devstack@keystone.service[81500]: Sep 25 01:00:24.435282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16075350-9e89-49c4-af3f-440dcda33edf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.456614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daefe5-2a87-4b5d-985f-f8b3f59d14c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.457828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daefe5-2a87-4b5d-985f-f8b3f59d14c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.458233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daefe5-2a87-4b5d-985f-f8b3f59d14c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.458559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daefe5-2a87-4b5d-985f-f8b3f59d14c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.459729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daefe5-2a87-4b5d-985f-f8b3f59d14c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.460940 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-17daefe5-2a87-4b5d-985f-f8b3f59d14c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 01:00:24.460940 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 01:00:24.460940 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 01:00:24.462132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daefe5-2a87-4b5d-985f-f8b3f59d14c5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.478021 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 777/1547] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/users => generated 284 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:00:24.480914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b33c7-f6ef-4a04-bb95-331de7878222 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.481984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b33c7-f6ef-4a04-bb95-331de7878222 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.482768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b33c7-f6ef-4a04-bb95-331de7878222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.514529 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 771/1548] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/projects => generated 377 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:24.515655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d3b4-e652-4422-995b-21f96c28a546 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.515987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d3b4-e652-4422-995b-21f96c28a546 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.520616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d3b4-e652-4422-995b-21f96c28a546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.525295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e32b33c7-f6ef-4a04-bb95-331de7878222 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.528147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e32b33c7-f6ef-4a04-bb95-331de7878222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.545998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5b58d3b4-e652-4422-995b-21f96c28a546 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.547300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5b58d3b4-e652-4422-995b-21f96c28a546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.624092 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 778/1549] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.625986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35724913-c600-4dbc-92ea-1c7394d48b60 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.626275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35724913-c600-4dbc-92ea-1c7394d48b60 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.626539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35724913-c600-4dbc-92ea-1c7394d48b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.634035 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 772/1550] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.636821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490db1ed-5621-45dd-b33b-c83ae5fe1d22 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.637699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490db1ed-5621-45dd-b33b-c83ae5fe1d22 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.638186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490db1ed-5621-45dd-b33b-c83ae5fe1d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.659418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-35724913-c600-4dbc-92ea-1c7394d48b60 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.660885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-35724913-c600-4dbc-92ea-1c7394d48b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.670024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-490db1ed-5621-45dd-b33b-c83ae5fe1d22 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.670330 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 779/1551] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.670832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35724913-c600-4dbc-92ea-1c7394d48b60 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.671370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-490db1ed-5621-45dd-b33b-c83ae5fe1d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.682522 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 773/1552] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.683506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490db1ed-5621-45dd-b33b-c83ae5fe1d22 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.690607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35724913-c600-4dbc-92ea-1c7394d48b60 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.698800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de82ff-206f-44bc-9c69-a8469620d162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.699078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490db1ed-5621-45dd-b33b-c83ae5fe1d22 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.699875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de82ff-206f-44bc-9c69-a8469620d162 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.700397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de82ff-206f-44bc-9c69-a8469620d162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.700907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de82ff-206f-44bc-9c69-a8469620d162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.702124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de82ff-206f-44bc-9c69-a8469620d162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.704610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afff8cb-6c57-4226-ae6c-f73cbcd94d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.705237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff8cb-6c57-4226-ae6c-f73cbcd94d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.705395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de82ff-206f-44bc-9c69-a8469620d162 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.705694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff8cb-6c57-4226-ae6c-f73cbcd94d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.705800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff8cb-6c57-4226-ae6c-f73cbcd94d9e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.706412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afff8cb-6c57-4226-ae6c-f73cbcd94d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.708750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afff8cb-6c57-4226-ae6c-f73cbcd94d9e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.727283 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 780/1553] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:24.729675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583b8b5-bb0c-40d5-a80a-42521ff34895 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.730168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583b8b5-bb0c-40d5-a80a-42521ff34895 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.730662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583b8b5-bb0c-40d5-a80a-42521ff34895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.744473 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 774/1554] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/users => generated 289 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:24.747785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c44af-2b66-42ef-accf-60205bca5c5c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.748443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c44af-2b66-42ef-accf-60205bca5c5c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.748893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c44af-2b66-42ef-accf-60205bca5c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.765287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0583b8b5-bb0c-40d5-a80a-42521ff34895 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.767636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0583b8b5-bb0c-40d5-a80a-42521ff34895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.777284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9b4c44af-2b66-42ef-accf-60205bca5c5c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.779034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b4c44af-2b66-42ef-accf-60205bca5c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.782895 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 781/1555] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.786265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6f29-8adf-4c45-a1c1-81dc7b778a41 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.786760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6f29-8adf-4c45-a1c1-81dc7b778a41 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.787246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6f29-8adf-4c45-a1c1-81dc7b778a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.791709 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 775/1556] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.793830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579f47-24db-436e-b78a-6f72b69bd68d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.794753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579f47-24db-436e-b78a-6f72b69bd68d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.795399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579f47-24db-436e-b78a-6f72b69bd68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.822377 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-68579f47-24db-436e-b78a-6f72b69bd68d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.823981 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68579f47-24db-436e-b78a-6f72b69bd68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.825159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-af3f6f29-8adf-4c45-a1c1-81dc7b778a41 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.827712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af3f6f29-8adf-4c45-a1c1-81dc7b778a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.837037 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 776/1557] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.838543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1830d16-8117-4d86-a1aa-9e2f73d8772a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.838822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1830d16-8117-4d86-a1aa-9e2f73d8772a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.838988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1830d16-8117-4d86-a1aa-9e2f73d8772a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.846274 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 782/1558] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.848842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f5f54-1968-4d80-862d-d6287995a88c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.849134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f5f54-1968-4d80-862d-d6287995a88c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.849473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f5f54-1968-4d80-862d-d6287995a88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.862818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b1830d16-8117-4d86-a1aa-9e2f73d8772a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.864063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b1830d16-8117-4d86-a1aa-9e2f73d8772a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.871784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e94f5f54-1968-4d80-862d-d6287995a88c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.873066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e94f5f54-1968-4d80-862d-d6287995a88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.873387 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 777/1559] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.874562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1830d16-8117-4d86-a1aa-9e2f73d8772a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.885086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1830d16-8117-4d86-a1aa-9e2f73d8772a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.890157 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 783/1560] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.890498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31364f19-0878-477c-9b92-08e9d1a2bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.891110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364f19-0878-477c-9b92-08e9d1a2bfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.891446 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364f19-0878-477c-9b92-08e9d1a2bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.891796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f5f54-1968-4d80-862d-d6287995a88c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.891999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364f19-0878-477c-9b92-08e9d1a2bfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.892757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31364f19-0878-477c-9b92-08e9d1a2bfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.894661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31364f19-0878-477c-9b92-08e9d1a2bfe3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.905114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:24.910750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f5f54-1968-4d80-862d-d6287995a88c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.913535 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 778/1561] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:24.917145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a61c3-1741-4e1b-b679-7704bfbea0d1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.917471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a61c3-1741-4e1b-b679-7704bfbea0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.917659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a61c3-1741-4e1b-b679-7704bfbea0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.919020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342ba97-fb3d-43a9-a41f-da8c5a71db03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.919637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342ba97-fb3d-43a9-a41f-da8c5a71db03 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.920861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342ba97-fb3d-43a9-a41f-da8c5a71db03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.921083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342ba97-fb3d-43a9-a41f-da8c5a71db03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.922058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342ba97-fb3d-43a9-a41f-da8c5a71db03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.923771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342ba97-fb3d-43a9-a41f-da8c5a71db03 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:24.932006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:24.936105 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 784/1562] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:24.937272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2dd2b-de8a-4ea0-bfe7-661cbea8ecd9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.937578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2dd2b-de8a-4ea0-bfe7-661cbea8ecd9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.937855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2dd2b-de8a-4ea0-bfe7-661cbea8ecd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.948730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cf6a61c3-1741-4e1b-b679-7704bfbea0d1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.951334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cf6a61c3-1741-4e1b-b679-7704bfbea0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.961569 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 779/1563] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.962576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a61c3-1741-4e1b-b679-7704bfbea0d1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.969687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-28f2dd2b-de8a-4ea0-bfe7-661cbea8ecd9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:24.972061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-28f2dd2b-de8a-4ea0-bfe7-661cbea8ecd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:24.980882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a61c3-1741-4e1b-b679-7704bfbea0d1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:24.985731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:24.989506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:24.989771 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 785/1564] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:24.989955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:24.989955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:24.989955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:24.989955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:24.990541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2dd2b-de8a-4ea0-bfe7-661cbea8ecd9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:24.990699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.005195 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 780/1565] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:25.005894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.009641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2dd2b-de8a-4ea0-bfe7-661cbea8ecd9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.019810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.020482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.020772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.021043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.021710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:25.022171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.023900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.025573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e40d5-f1aa-45d1-8914-d9ab25fc99a9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.036184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.037258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.037767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.038127 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 786/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:25.038305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.039248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.039495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:25.040040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.052260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.067434 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 781/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:25.068237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ccaf8-23f7-4b5f-ab7b-613b3c1e5dec admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.069508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.077995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.079220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.079506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.079976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.081250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:25.081801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.084883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.088138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea953d-c191-4615-b1d9-8c8c2a29183b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.096190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.097385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.097891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.098979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] PATH_INFO: `/v3/projects/b4ae3650ef5943a59c8c8485d1a9b983/users/291c2d9f7b1e40db8bb5c34dcc33d9d8/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.111009 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 787/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:25.112251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.127880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e79d46-fecd-4975-b3a2-345320059d20 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.135289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ae3650ef5943a59c8c8485d1a9b983, user_id=291c2d9f7b1e40db8bb5c34dcc33d9d8, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.137259 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 01:00:25.137259 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 01:00:25.137259 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 01:00:25.137621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.138355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.138844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.139329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.140627 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.141262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.144007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.163865 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 782/1569] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:25 2022] PUT /identity/v3/projects/b4ae3650ef5943a59c8c8485d1a9b983/users/291c2d9f7b1e40db8bb5c34dcc33d9d8/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:25.165788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.170132 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 788/1570] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:25.170132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.187873 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fdcf4-3116-4550-b41a-5e0ef138379b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.264200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4b76c-b684-4a65-a313-83e3f79f7d72 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.274187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.275250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.275774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.276261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.277831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:25.278629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.281849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.283850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.284523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.284808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.285129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.285756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:25.286064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.287794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.301386 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 783/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:25.307896 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 789/1572] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:25.309469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.311688 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.325885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdd017-1bfc-4e56-8aab-b8819a4478a4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.327781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb542f-0d4f-4d4f-a9fc-0d82cf74bc4e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.331561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.332468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.332752 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.333036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.333250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.333655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.333823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.333971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.334228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.334946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.336034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.337414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.381283 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 790/1573] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:25.381283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.395135 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 784/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:25.396718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.472700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239460f-7e03-484c-8837-9838af88de18 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.513785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:25.518117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd0a28-eec0-4f97-a252-c125872f9fdc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.526228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.526228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.526861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.526861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.527442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.529958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.557819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486d3ac-68f0-4554-8690-3bbbdaf4e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.558692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d3ac-68f0-4554-8690-3bbbdaf4e01c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.559135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d3ac-68f0-4554-8690-3bbbdaf4e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.559612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486d3ac-68f0-4554-8690-3bbbdaf4e01c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.560469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486d3ac-68f0-4554-8690-3bbbdaf4e01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.562828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486d3ac-68f0-4554-8690-3bbbdaf4e01c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.572972 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 785/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/projects => generated 393 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:25.572972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.578030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:25.617557 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 791/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/projects => generated 389 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:25.619313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5e67-7adc-4efb-b849-32a8a276f659 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.619801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5e67-7adc-4efb-b849-32a8a276f659 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.620266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5e67-7adc-4efb-b849-32a8a276f659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.661718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dd8c5e67-7adc-4efb-b849-32a8a276f659 None None] MFA Rules not processed for user `291c2d9f7b1e40db8bb5c34dcc33d9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:25.663207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dd8c5e67-7adc-4efb-b849-32a8a276f659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:25.670315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baaed0-6b5f-43ca-81a7-6cdfee7874d1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.767541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.768869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.769163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.769444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] PATH_INFO: `/v3/projects/537d0a114f314e9b948150001bda642c/users/f32c9b07e6234d55b0d4846d44ee6ce6/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.774360 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 792/1577] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:25.775514 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c5e67-7adc-4efb-b849-32a8a276f659 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.788325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537d0a114f314e9b948150001bda642c, user_id=f32c9b07e6234d55b0d4846d44ee6ce6, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.791710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.792756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c5e67-7adc-4efb-b849-32a8a276f659 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.803705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.805431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.805939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.806433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.807686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.811749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.822873 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 786/1578] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:25 2022] PUT /identity/v3/projects/537d0a114f314e9b948150001bda642c/users/f32c9b07e6234d55b0d4846d44ee6ce6/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:25.823751 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.859181 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 793/1579] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/projects => generated 403 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:25.861021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:25.919680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fc756-d641-4250-8cb2-c64eb6a4f9e4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.985363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad55c8f-8456-46ac-b745-e23139447aa8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:25.990776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:25.991410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:25.992184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:25.992494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:25.993139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:25.995152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:25.999957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:26.025756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.026299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.030661 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 794/1580] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:26.030800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.030800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.030800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.031049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.031135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.048643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27861056-eccb-47fc-8057-11d84dbb85f8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.059106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.060238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.060756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.061281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.062421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.066212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.072187 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 787/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:25 2022] POST /identity/v3/projects => generated 403 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:26.072187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.081616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:26.098679 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 795/1582] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:26 2022] POST /identity/v3/users => generated 291 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:26.100244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.118986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4349-0d76-4f82-bda9-1ffe454d3764 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.189325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b4521-08de-420f-96ba-16bc3c1b11b7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.200186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.201047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.201470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.201876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.202653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.206333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.221326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cae17-05ab-4ab5-b270-d0216c44716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.222827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cae17-05ab-4ab5-b270-d0216c44716b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.223397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cae17-05ab-4ab5-b270-d0216c44716b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.224114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cae17-05ab-4ab5-b270-d0216c44716b admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.225667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cae17-05ab-4ab5-b270-d0216c44716b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.230163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cae17-05ab-4ab5-b270-d0216c44716b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.251563 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 788/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:26 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:26.253398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.268999 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 796/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:26 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:26.268999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628e279-5d74-4122-836a-dfd0731232d8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.268999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628e279-5d74-4122-836a-dfd0731232d8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.268999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628e279-5d74-4122-836a-dfd0731232d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.292860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7628e279-5d74-4122-836a-dfd0731232d8 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:26.294046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7628e279-5d74-4122-836a-dfd0731232d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:26.305443 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 797/1585] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:26.306391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628e279-5d74-4122-836a-dfd0731232d8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.318480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628e279-5d74-4122-836a-dfd0731232d8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.325065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.325715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.325997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.326269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.326941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.329012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.362225 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 798/1586] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:26 2022] POST /identity/v3/users => generated 302 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:26.363716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.375079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cd84f-6ba1-460c-b5e9-87fe6b0ff15d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.383184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.384286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.384825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.385425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.387838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef0281-7f7f-4f0c-8687-ec5ba57b4e0c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.395863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.396891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.397358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.397864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.399535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.402179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.416341 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 799/1587] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.417549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.429491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69874e71-8477-4abb-950a-d6e3ffcd1c29 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.435096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.435632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.435947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.436171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.436964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.439117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.470995 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 800/1588] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:26 2022] POST /identity/v3/users => generated 302 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:26.472092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.475356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.478800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.485075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d1af0-727d-40a2-ad9a-5dcfca42bd50 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.491396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.492968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.493432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.494042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.495160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.498328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.499293 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 789/1589] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:26.501210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.521952 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 801/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.523668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47f1ec-59d0-4677-b01d-93805174a6da service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.527080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.533238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.534291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.534801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.535275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.536519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.539779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.540166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79625c76-1e2b-49dd-af16-d3e0cb5d7b10 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.547663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.548341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.550925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:26.552303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.552303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.553855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.556512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.562579 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 790/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.563843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.576258 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 802/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.576951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.591361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974e329-a981-4191-8dc3-1181666f8a52 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.598248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.599422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.599856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.600198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.601141 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.602885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.615538 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 803/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.616554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.641572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8aab0-341e-41bb-8089-96cd4e9fba21 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.646831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.647625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.647976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.648776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.649444 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.651197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.663583 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 804/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.668083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.669172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb15add-acda-4cdb-a1ee-16ff9ed72733 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.678466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.679575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14168c7a-a3ff-48a0-82c0-89fc9c71552b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.679831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.680358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.680985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.684882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.685915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.686206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.686481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.687295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.689410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.699532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.703268 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 805/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.703473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.704058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.722201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208b123-9e30-49ce-88a5-eeb2fc3c05da admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.727831 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 791/1596] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:26.728617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.729428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.729792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.730236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.730854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.731884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.734790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.749231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a948b4-3159-4439-a4fd-282a3c194e8f service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.754063 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 806/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.756788 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.765433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.765788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.765788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.766127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.766745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.768609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.773949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abad4e-542d-4aae-bafb-3ad858b04cb5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.781476 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 792/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.781782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.782047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.782232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.782480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.783281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.786059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.786376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.802949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26f05-e365-4eab-832b-ad7d0e2ac549 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.809163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.809979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.810365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.810715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.811452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.813292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.819163 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 807/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.823068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.828542 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 793/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.829672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.841935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ccd0-5c7b-43fd-973f-27a2dc7bf7fa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.847882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.848649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.849007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.849596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.850484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.852264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530ebb8-8cb9-448a-a31c-26cf556f4e07 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.852751 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.869006 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 794/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:26.871323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.872524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.873049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.873506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] PATH_INFO: `/v3/projects/0b369e7845314685b4145d3feaa02258/users/ef66ca98e383463994017e057c499c6f/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.875691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.891690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f9be2-98d2-4aed-8c6c-cb239ff013cd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:26.897186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:26.898064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:26.898547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:26.898958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] PATH_INFO: `/v3/projects/14301d7630d542c3ba3f008820385459/users/d6614e10f6384972a1d3d7485a7a9124/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:26.911141 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:26.924500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14301d7630d542c3ba3f008820385459, user_id=d6614e10f6384972a1d3d7485a7a9124, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.925883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b369e7845314685b4145d3feaa02258, user_id=ef66ca98e383463994017e057c499c6f, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:26.928926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.939740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:26.952998 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 795/1602] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:26 2022] PUT /identity/v3/projects/14301d7630d542c3ba3f008820385459/users/d6614e10f6384972a1d3d7485a7a9124/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:26.965922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:26.975338 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 808/1603] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:26 2022] PUT /identity/v3/projects/0b369e7845314685b4145d3feaa02258/users/ef66ca98e383463994017e057c499c6f/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:26.980596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:27.117184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d3fd3-c2db-4cf9-ab11-e017e2167e23 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:27.177471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c52d7-6216-48b5-9616-f91351632c64 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:27.188968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:27.190463 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:27.191053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:27.192310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] PATH_INFO: `/v3/projects/cc51ee9f630a4bc89f1fccbd7ff20c51/users/c468aa6d1248478286fe0a8792411a91/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:27.203296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:27.224762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:27.225934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:27.226461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:27.228922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] PATH_INFO: `/v3/projects/5b11c2ffde89471cafaf35f8579e3ab2/users/ca20e105b76149dd99e4aa64c62eb483/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:27.236759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc51ee9f630a4bc89f1fccbd7ff20c51, user_id=c468aa6d1248478286fe0a8792411a91, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:27.241967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:27.251807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b11c2ffde89471cafaf35f8579e3ab2, user_id=ca20e105b76149dd99e4aa64c62eb483, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:27.255990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:27.280640 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 809/1604] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:26 2022] PUT /identity/v3/projects/cc51ee9f630a4bc89f1fccbd7ff20c51/users/c468aa6d1248478286fe0a8792411a91/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:27.281852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:27.297622 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 796/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:26 2022] PUT /identity/v3/projects/5b11c2ffde89471cafaf35f8579e3ab2/users/ca20e105b76149dd99e4aa64c62eb483/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:27.309204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:27.449472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88947392-be0a-44fd-b3cf-57e653369f82 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:27.505075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be8ba1-b223-468f-b95c-0c6ada5f9c71 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:27.532404 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21473ac2-4158-46cf-a70a-07ed18c146f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:27.534028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473ac2-4158-46cf-a70a-07ed18c146f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:27.537117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473ac2-4158-46cf-a70a-07ed18c146f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:27.537117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473ac2-4158-46cf-a70a-07ed18c146f2 admin admin] PATH_INFO: `/v3/projects/6535f45ac68f4051973150a563e8db0d/users/e20eebb25c5d4af1b39bee09f1bfb9c1/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:27.598660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21473ac2-4158-46cf-a70a-07ed18c146f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6535f45ac68f4051973150a563e8db0d, user_id=e20eebb25c5d4af1b39bee09f1bfb9c1, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:27.604201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21473ac2-4158-46cf-a70a-07ed18c146f2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:27.636798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c3bb9-761f-43a1-bc1c-74b24cb4ab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:27.637823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3bb9-761f-43a1-bc1c-74b24cb4ab7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:27.637823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3bb9-761f-43a1-bc1c-74b24cb4ab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:27.638291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3bb9-761f-43a1-bc1c-74b24cb4ab7d admin admin] PATH_INFO: `/v3/projects/eed58e41757d45c4b4d197b631fefb91/users/33515af005b744e6897c2de60f4f73dd/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:27.647587 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 797/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:27 2022] PUT /identity/v3/projects/6535f45ac68f4051973150a563e8db0d/users/e20eebb25c5d4af1b39bee09f1bfb9c1/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:27.650077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4891-c218-49d1-a9be-877c28c27e26 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:27.650682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4891-c218-49d1-a9be-877c28c27e26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:27.651334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4891-c218-49d1-a9be-877c28c27e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:27.694455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c3bb9-761f-43a1-bc1c-74b24cb4ab7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed58e41757d45c4b4d197b631fefb91, user_id=33515af005b744e6897c2de60f4f73dd, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:27.710146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c3bb9-761f-43a1-bc1c-74b24cb4ab7d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:27.720040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-142d4891-c218-49d1-a9be-877c28c27e26 None None] MFA Rules not processed for user `d6614e10f6384972a1d3d7485a7a9124`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:27.721979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-142d4891-c218-49d1-a9be-877c28c27e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:27.746877 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 810/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:27 2022] PUT /identity/v3/projects/eed58e41757d45c4b4d197b631fefb91/users/33515af005b744e6897c2de60f4f73dd/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:27.769830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6af3b-078f-4468-a567-4bcbd28cc096 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:27.775000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6af3b-078f-4468-a567-4bcbd28cc096 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:27.775739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6af3b-078f-4468-a567-4bcbd28cc096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:27.790488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:27.820127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7cd6af3b-078f-4468-a567-4bcbd28cc096 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:27.823015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7cd6af3b-078f-4468-a567-4bcbd28cc096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:27.906388 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 798/1608] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:27.910802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511c6bf-3e37-46c3-934e-4253c114a70c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:27.911266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511c6bf-3e37-46c3-934e-4253c114a70c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:27.911738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511c6bf-3e37-46c3-934e-4253c114a70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:27.952777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f511c6bf-3e37-46c3-934e-4253c114a70c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:27.955075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f511c6bf-3e37-46c3-934e-4253c114a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:28.002645 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 811/1609] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:28.008430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287abfcc-7ccd-4172-9316-e3202947d4d7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.008991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287abfcc-7ccd-4172-9316-e3202947d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.009479 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287abfcc-7ccd-4172-9316-e3202947d4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.057834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-287abfcc-7ccd-4172-9316-e3202947d4d7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:28.060341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-287abfcc-7ccd-4172-9316-e3202947d4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:28.063987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:28.079518 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 812/1610] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:28.079518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718543cc-5790-4ea7-9e1c-eaaf945eac18 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.079518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718543cc-5790-4ea7-9e1c-eaaf945eac18 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.080080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718543cc-5790-4ea7-9e1c-eaaf945eac18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.104102 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 799/1611] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:28.105613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2dac5-8212-44fb-8325-4c2d51e07164 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.105910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2dac5-8212-44fb-8325-4c2d51e07164 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.106420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2dac5-8212-44fb-8325-4c2d51e07164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.127141 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-718543cc-5790-4ea7-9e1c-eaaf945eac18 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:28.129927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-718543cc-5790-4ea7-9e1c-eaaf945eac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:28.151619 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 813/1612] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:28.151619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467134-b999-49a3-a85c-36711546f208 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.151952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467134-b999-49a3-a85c-36711546f208 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.152420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467134-b999-49a3-a85c-36711546f208 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.160592 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 814/1613] 10.222.0.26 () {54 vars in 961 bytes} [Sun Sep 25 01:00:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:28.164269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2467134-b999-49a3-a85c-36711546f208 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:28.174283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9df2dac5-8212-44fb-8325-4c2d51e07164 None None] MFA Rules not processed for user `33515af005b744e6897c2de60f4f73dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:28.176124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9df2dac5-8212-44fb-8325-4c2d51e07164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:28.268057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2467134-b999-49a3-a85c-36711546f208 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:28.279734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:28.281670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.282337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.282922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.320698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:28.324676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:28.351507 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 800/1614] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:28.353046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2dac5-8212-44fb-8325-4c2d51e07164 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:28.358299 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 815/1615] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:28.361556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:28.388331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73506a7e-48d2-48bf-97f7-d909f6634598 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:28.392449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2dac5-8212-44fb-8325-4c2d51e07164 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:28.398785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ea184-0914-404c-a41e-a4db84befe67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:28.400743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea184-0914-404c-a41e-a4db84befe67 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.401745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea184-0914-404c-a41e-a4db84befe67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.402378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea184-0914-404c-a41e-a4db84befe67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.403976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:28.405947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.408493 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.409406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.435274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ea184-0914-404c-a41e-a4db84befe67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:28.438578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ea184-0914-404c-a41e-a4db84befe67 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:28.447253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:28.449420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:28.462618 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 816/1616] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:28.467068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6eb89-55f7-4de6-82b0-1106c49fb734 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.471404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6eb89-55f7-4de6-82b0-1106c49fb734 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.471855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6eb89-55f7-4de6-82b0-1106c49fb734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.480582 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 801/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:28.481708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:28.510825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39aa9ea-1c5c-4273-b969-4140e0606c1a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:28.517612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:28.518481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.518956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.519341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.533390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:28.535184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:28.561290 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 802/1618] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:28.561894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-adf6eb89-55f7-4de6-82b0-1106c49fb734 None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:28.564337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-adf6eb89-55f7-4de6-82b0-1106c49fb734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:28.564672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:28.602773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ffac2-3b1c-4f43-9dab-617080d928ef service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:28.625928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:28.629135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:28.629760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:28.630253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:28.635278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:28.638761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:28.702177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:28.736436 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 817/1619] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Sep 25 01:00:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:28.742356 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 803/1620] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:28 2022] POST /identity/v3/projects => generated 405 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:28.747079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6eb89-55f7-4de6-82b0-1106c49fb734 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:28.761198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:28.782491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6eb89-55f7-4de6-82b0-1106c49fb734 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:29.033980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:29.035455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.037984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.038693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.041439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:29.045220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:29.050541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a2396-0f67-403e-a4f6-d1ea6a3f7f30 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:29.067133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:29.068781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.069298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.069884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.125208 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 818/1621] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:28 2022] POST /identity/v3/users => generated 309 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:29.131698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:29.154935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a701-6bc3-4c5f-8b6c-a8f9b7954206 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:29.167235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:29.171948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:29.173216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.173702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.174384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.177345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:29.179799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:29.221602 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 819/1622] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:00:29.223173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:29.249244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d42298-8d53-4901-aff0-aa517165b831 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:29.264432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:29.266442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:29.269415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:29.270352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.271589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.272652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] PATH_INFO: `/v3/projects/f19fc5babed04f21a1f2ca9fe3161bb1/users/ec904ac7942b49b3ba7a8aa883f72bfb/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.296900 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 804/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:29.318469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19fc5babed04f21a1f2ca9fe3161bb1, user_id=ec904ac7942b49b3ba7a8aa883f72bfb, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:29.324619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:29.357888 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 820/1624] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:29 2022] PUT /identity/v3/projects/f19fc5babed04f21a1f2ca9fe3161bb1/users/ec904ac7942b49b3ba7a8aa883f72bfb/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:29.363369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:29.388167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fee689-9170-4c80-821c-d084817cad2e service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:29.568135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac317fd2-ec09-42d8-86ba-58659e1756b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:29.569307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac317fd2-ec09-42d8-86ba-58659e1756b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.569564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac317fd2-ec09-42d8-86ba-58659e1756b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.570299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac317fd2-ec09-42d8-86ba-58659e1756b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.576926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac317fd2-ec09-42d8-86ba-58659e1756b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:29.580121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac317fd2-ec09-42d8-86ba-58659e1756b0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:29.625891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:29.640860 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 805/1625] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:00:29.660061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0507-5e83-4c84-822a-e2640bab8754 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:29.679699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c57cd-72b1-45ec-9125-c84e25c79b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:29.680899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c57cd-72b1-45ec-9125-c84e25c79b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.681399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c57cd-72b1-45ec-9125-c84e25c79b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.681865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c57cd-72b1-45ec-9125-c84e25c79b52 admin admin] PATH_INFO: `/v3/projects/f19fc5babed04f21a1f2ca9fe3161bb1/users/ec904ac7942b49b3ba7a8aa883f72bfb/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.688250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c57cd-72b1-45ec-9125-c84e25c79b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19fc5babed04f21a1f2ca9fe3161bb1, user_id=ec904ac7942b49b3ba7a8aa883f72bfb, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:29.693335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c57cd-72b1-45ec-9125-c84e25c79b52 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:29.742242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0397d-5ff2-4b0a-951c-863700383ef4 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.742730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0397d-5ff2-4b0a-951c-863700383ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.743216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0397d-5ff2-4b0a-951c-863700383ef4 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.746454 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 806/1626] 10.222.0.26 () {54 vars in 961 bytes} [Sun Sep 25 01:00:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:29.749275 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 821/1627] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:29 2022] PUT /identity/v3/projects/f19fc5babed04f21a1f2ca9fe3161bb1/users/ec904ac7942b49b3ba7a8aa883f72bfb/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:29.770019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff37447-74dd-4dc2-bbd2-81aee42f0ba4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.770462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff37447-74dd-4dc2-bbd2-81aee42f0ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.770920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff37447-74dd-4dc2-bbd2-81aee42f0ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.802451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d538c7-49d0-4578-b125-3f0a4279f44d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:29.802793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d538c7-49d0-4578-b125-3f0a4279f44d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:29.803130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d538c7-49d0-4578-b125-3f0a4279f44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:29.857860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-91d538c7-49d0-4578-b125-3f0a4279f44d None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:29.859159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-91d538c7-49d0-4578-b125-3f0a4279f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:29.862878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8ff37447-74dd-4dc2-bbd2-81aee42f0ba4 None None] MFA Rules not processed for user `ec904ac7942b49b3ba7a8aa883f72bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:29.864132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8ff37447-74dd-4dc2-bbd2-81aee42f0ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:30.051624 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 822/1628] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Sep 25 01:00:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:30.064518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0712d0-93e0-439f-bd2c-91f83e4ebe84 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:30.064755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0712d0-93e0-439f-bd2c-91f83e4ebe84 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:30.065056 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0712d0-93e0-439f-bd2c-91f83e4ebe84 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:30.070616 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 823/1629] 10.222.0.26 () {54 vars in 912 bytes} [Sun Sep 25 01:00:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:30.090030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d246b-fd70-4cb6-af26-1d6a6e3ee2a9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:30.090732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d246b-fd70-4cb6-af26-1d6a6e3ee2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:30.090732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d246b-fd70-4cb6-af26-1d6a6e3ee2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:30.146103 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 807/1630] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:30.150428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-161d246b-fd70-4cb6-af26-1d6a6e3ee2a9 None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:30.153934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-161d246b-fd70-4cb6-af26-1d6a6e3ee2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:30.158099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff37447-74dd-4dc2-bbd2-81aee42f0ba4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:30.208454 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 824/1631] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:00:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:30.301617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d246b-fd70-4cb6-af26-1d6a6e3ee2a9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:30.323323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:30.330858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff37447-74dd-4dc2-bbd2-81aee42f0ba4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:30.336317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd3c0f-222c-4664-a343-7235632015a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:30.337349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd3c0f-222c-4664-a343-7235632015a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:30.337482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd3c0f-222c-4664-a343-7235632015a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:30.337830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd3c0f-222c-4664-a343-7235632015a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:30.341348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:30.371203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd3c0f-222c-4664-a343-7235632015a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:30.373928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd3c0f-222c-4664-a343-7235632015a5 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:30.399188 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 808/1632] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:30 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:00:30.401252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4e6af-69e0-45ff-90be-2861c23e955a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:30.401542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4e6af-69e0-45ff-90be-2861c23e955a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:30.413094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4e6af-69e0-45ff-90be-2861c23e955a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:30.413094 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 809/1633] 10.222.0.26 () {54 vars in 912 bytes} [Sun Sep 25 01:00:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:30.418449 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10449353-9fd0-4c3d-beb4-e4261ad6bfbc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:30.419512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10449353-9fd0-4c3d-beb4-e4261ad6bfbc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:30.419801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10449353-9fd0-4c3d-beb4-e4261ad6bfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:30.479753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-10449353-9fd0-4c3d-beb4-e4261ad6bfbc None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:30.482063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-10449353-9fd0-4c3d-beb4-e4261ad6bfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:30.508213 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 810/1634] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:00:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:30.520000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d246b-fd70-4cb6-af26-1d6a6e3ee2a9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:30.538853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:30.539527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:30.540196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:30.540889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:30.542950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10449353-9fd0-4c3d-beb4-e4261ad6bfbc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:30.556882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10449353-9fd0-4c3d-beb4-e4261ad6bfbc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:30.563724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8223c9-a6f3-43cc-b3fe-bb6e3b745979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:30.565023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8223c9-a6f3-43cc-b3fe-bb6e3b745979 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:30.565621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8223c9-a6f3-43cc-b3fe-bb6e3b745979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:30.566052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8223c9-a6f3-43cc-b3fe-bb6e3b745979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:30.566572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:30.569878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:30.584871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8223c9-a6f3-43cc-b3fe-bb6e3b745979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:30.587789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8223c9-a6f3-43cc-b3fe-bb6e3b745979 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:30.594251 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 825/1635] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:30.617316 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 811/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:31.768276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:32.119047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e2dd-1b91-4ba2-8664-dfd000241c80 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:32.144636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:32.145698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:32.146185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:32.146663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] PATH_INFO: `/v3/users/33515af005b744e6897c2de60f4f73dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:32.158348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33515af005b744e6897c2de60f4f73dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:32.163168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:32.379950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33515af005b744e6897c2de60f4f73dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:32.384042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33515af005b744e6897c2de60f4f73dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:32.407299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33515af005b744e6897c2de60f4f73dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:32.441212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33515af005b744e6897c2de60f4f73dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:32.443427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9c61ee42-c7d4-4203-8f03-2ec68aeb96d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33515af005b744e6897c2de60f4f73dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:32.559190 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 826/1637] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:31 2022] DELETE /identity/v3/users/33515af005b744e6897c2de60f4f73dd => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:32.567371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:32.567929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:32.568751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:32.568751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:32.568949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:32.571507 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:32.613806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:32.617439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:32.634980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] MFA Rules not processed for user `e20eebb25c5d4af1b39bee09f1bfb9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:32.639321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:32.758236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:32.853567 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 812/1638] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:32.874376 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 827/1639] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:32 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:32.876448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:32.879644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:32.932856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e4a55-d075-404f-a09e-69a3eacce97e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:32.954556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:32.955665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:32.956169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:32.956715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:32.957937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:32.961971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:32.970754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:32.973415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:33.068605 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 813/1640] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:32 2022] POST /identity/v3/projects => generated 403 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:33.075703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:33.181264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdca0a8-2b68-4461-a857-2f45139f5cfc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:33.252191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e1ed9-221c-4e41-8152-f3a75361b3c5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:33.272405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:33.275985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:33.278556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:33.278977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:33.282003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:33.285374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:33.373530 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 814/1641] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:33 2022] POST /identity/v3/users => generated 308 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:00:33.380602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:33.396566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4c447-d10a-4704-96c5-39fe14f4e80e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:33.412611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:33.414366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:33.414366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:33.414886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:33.415554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:33.435138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:33.469848 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 815/1642] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:33.477008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:33.504923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b15c4-f0cf-4403-a13f-9f64b08412eb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:33.533694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:33.534760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:33.535287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:33.535686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:33.541402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:33.542569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:33.543073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:33.543567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] PATH_INFO: `/v3/projects/68a28b8b67e34eb08d2ace41e1a947d9/users/acf570b732d045ad86af4700b999a3bb/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:33.585146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a28b8b67e34eb08d2ace41e1a947d9, user_id=acf570b732d045ad86af4700b999a3bb, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:33.589336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:33.595426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:33.603717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:33.647226 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 816/1643] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:33 2022] PUT /identity/v3/projects/68a28b8b67e34eb08d2ace41e1a947d9/users/acf570b732d045ad86af4700b999a3bb/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:33.648191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:33.661026 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 828/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:00:33.664098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:33.922772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac23e49-11d6-47e4-8b75-e9728ee73f0f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:33.943120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:33.944558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:33.945327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:33.946042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:33.947428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:33.950868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:34.000748 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 817/1645] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:34.026338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:34.070920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb238e7-7fbf-4312-94ff-184c92e342cc service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:34.099832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:34.103129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:34.103718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:34.104228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] PATH_INFO: `/v3/projects/68a28b8b67e34eb08d2ace41e1a947d9/users/acf570b732d045ad86af4700b999a3bb/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:34.118907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68a28b8b67e34eb08d2ace41e1a947d9, user_id=acf570b732d045ad86af4700b999a3bb, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:34.132299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:34.187367 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 829/1646] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:34 2022] PUT /identity/v3/projects/68a28b8b67e34eb08d2ace41e1a947d9/users/acf570b732d045ad86af4700b999a3bb/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:36.251213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:36.254353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:36.293599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650560a-f49c-4c1a-b366-b1abb6a8da29 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:36.456724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:36.457499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:36.457856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:36.458210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] PATH_INFO: `/v3/projects/eed58e41757d45c4b4d197b631fefb91` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:36.463712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed58e41757d45c4b4d197b631fefb91)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:36.465902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:36.622771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed58e41757d45c4b4d197b631fefb91', 'request_id': 'req-8802b409-65fb-4092-baa0-df60c36f9da7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:36.639367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed58e41757d45c4b4d197b631fefb91', 'request_id': 'req-8802b409-65fb-4092-baa0-df60c36f9da7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:36.659231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed58e41757d45c4b4d197b631fefb91', 'request_id': 'req-8802b409-65fb-4092-baa0-df60c36f9da7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:36.688311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] The token cache is being invalidate because project eed58e41757d45c4b4d197b631fefb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:36.688739 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:36.690572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:36.698432 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 818/1647] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:36 2022] DELETE /identity/v3/projects/eed58e41757d45c4b4d197b631fefb91 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:36.700294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:36.703023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:36.752213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceded1c-818f-40b5-b925-cdc32fc2067c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:36.928760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:36.929106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:36.930074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:36.930435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] PATH_INFO: `/v3/users/ec904ac7942b49b3ba7a8aa883f72bfb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:36.933523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec904ac7942b49b3ba7a8aa883f72bfb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:36.936541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:36.978859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:37.027861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec904ac7942b49b3ba7a8aa883f72bfb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:37.028824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec904ac7942b49b3ba7a8aa883f72bfb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:37.039562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec904ac7942b49b3ba7a8aa883f72bfb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:37.059201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec904ac7942b49b3ba7a8aa883f72bfb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:37.060013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2cc18d3-7440-4143-bf2b-65431778ab0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec904ac7942b49b3ba7a8aa883f72bfb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:37.100169 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 830/1648] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:36 2022] DELETE /identity/v3/users/ec904ac7942b49b3ba7a8aa883f72bfb => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:38.840096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:38.843354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:38.881894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b409-65fb-4092-baa0-df60c36f9da7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:39.036580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:39.155171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:39.158848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:39.159213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:39.159502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] PATH_INFO: `/v3/users/d6614e10f6384972a1d3d7485a7a9124` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:39.161881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6614e10f6384972a1d3d7485a7a9124)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:39.167148 dv-cinder-master-3-smb devstack@keystone.service[81499]: WARNING py.warnings [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 01:00:39.167148 dv-cinder-master-3-smb devstack@keystone.service[81499]: warnings.warn(msg) Sep 25 01:00:39.167148 dv-cinder-master-3-smb devstack@keystone.service[81499]: Sep 25 01:00:39.169935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:39.379702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6614e10f6384972a1d3d7485a7a9124'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:39.393477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6614e10f6384972a1d3d7485a7a9124'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:39.431279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6614e10f6384972a1d3d7485a7a9124'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:39.434606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6614e10f6384972a1d3d7485a7a9124'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:39.436520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8e5cf18f-440c-4154-a15c-f86bc40a7ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6614e10f6384972a1d3d7485a7a9124'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:39.464525 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 819/1649] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:38 2022] DELETE /identity/v3/users/d6614e10f6384972a1d3d7485a7a9124 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:39.471372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:39.472225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:39.472751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:39.571459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:39.581780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:39.622035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:39.622536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:39.623022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:39.769503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] MFA Rules not processed for user `acf570b732d045ad86af4700b999a3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:39.771268 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:40.005368 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 831/1650] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:40.104690 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 820/1651] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:39 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:40.139924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:40.144459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:40.376228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f41e-a335-4845-b74e-057cf7dd2248 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:40.402019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c43ac-de53-49a3-b2db-03766eb22b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:40.402675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c43ac-de53-49a3-b2db-03766eb22b05 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:40.403123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c43ac-de53-49a3-b2db-03766eb22b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:40.403853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c43ac-de53-49a3-b2db-03766eb22b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:40.446670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c43ac-de53-49a3-b2db-03766eb22b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:40.448338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c43ac-de53-49a3-b2db-03766eb22b05 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:40.512268 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 832/1652] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:41.217432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:41.220277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:41.263938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d681ac-8929-48da-82af-a6a2626104a1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:41.274768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:41.276074 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:41.276616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:41.277067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] PATH_INFO: `/v3/projects/f19fc5babed04f21a1f2ca9fe3161bb1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:41.279489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19fc5babed04f21a1f2ca9fe3161bb1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:41.287325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:41.454411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:41.582688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19fc5babed04f21a1f2ca9fe3161bb1', 'request_id': 'req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:41.618223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19fc5babed04f21a1f2ca9fe3161bb1', 'request_id': 'req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:41.633438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19fc5babed04f21a1f2ca9fe3161bb1', 'request_id': 'req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:41.666288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] The token cache is being invalidate because project f19fc5babed04f21a1f2ca9fe3161bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:41.666471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:41.667389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:41.672649 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 821/1653] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:41 2022] DELETE /identity/v3/projects/f19fc5babed04f21a1f2ca9fe3161bb1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:41.682156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf54eb-aa2f-4281-98bd-aa3ff8ffa7ce None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:41.682707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf54eb-aa2f-4281-98bd-aa3ff8ffa7ce None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:41.687107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf54eb-aa2f-4281-98bd-aa3ff8ffa7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:41.741152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:41.743757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:41.756838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a4bf54eb-aa2f-4281-98bd-aa3ff8ffa7ce None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:41.763265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a4bf54eb-aa2f-4281-98bd-aa3ff8ffa7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:41.897235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b922-8c8a-47c8-96bf-b7cfb363b72c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:41.908736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:41.911075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:41.912121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:41.912288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:41.912686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] PATH_INFO: `/v3/users/291c2d9f7b1e40db8bb5c34dcc33d9d8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:41.915280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=291c2d9f7b1e40db8bb5c34dcc33d9d8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:41.919789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:41.986564 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 833/1654] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:41.990463 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f8a0f-b130-4920-a5c3-58bd79ece402 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:41.990709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f8a0f-b130-4920-a5c3-58bd79ece402 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:41.991092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f8a0f-b130-4920-a5c3-58bd79ece402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:42.038066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b42f8a0f-b130-4920-a5c3-58bd79ece402 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:42.040936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291c2d9f7b1e40db8bb5c34dcc33d9d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:42.042835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b42f8a0f-b130-4920-a5c3-58bd79ece402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:42.066571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291c2d9f7b1e40db8bb5c34dcc33d9d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:42.079210 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 834/1655] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:42.080878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f8a0f-b130-4920-a5c3-58bd79ece402 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:42.098797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f8a0f-b130-4920-a5c3-58bd79ece402 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:42.099789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291c2d9f7b1e40db8bb5c34dcc33d9d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:42.107192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291c2d9f7b1e40db8bb5c34dcc33d9d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:42.107948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddd6e36c-fb48-4f15-a8ed-e7922d0328cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291c2d9f7b1e40db8bb5c34dcc33d9d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:42.110450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:42.113887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:42.114694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:42.115149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:42.116344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:42.116971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:42.120063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:42.139794 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 822/1656] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:41 2022] DELETE /identity/v3/users/291c2d9f7b1e40db8bb5c34dcc33d9d8 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:42.146992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:42.147295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:42.147561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:42.170160 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 835/1657] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:00:42.171944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:42.176766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:42.190201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:42.192303 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:42.208869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735fca3-7c16-49c0-9439-be67a9fc96c5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:42.313929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:42.379288 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 823/1658] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:42.452288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:42.453374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:42.454020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:42.454583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:42.455928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:42.455928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:42.456983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:42.462536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:42.590402 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 836/1659] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:42 2022] POST /identity/v3/projects => generated 381 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:42.592301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:42.622573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198f101-01bf-4666-9428-851dc23758c8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:42.721827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:42.722482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:42.722803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:42.723084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:42.801260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cbcc4c-7e68-4d5b-8008-93bbca8421f4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:42.814673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:42.815647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:42.816036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:42.816469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:42.819231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:42.823715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:42.847090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:42.850687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:42.867869 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 824/1660] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:00:42.875664 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 837/1661] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:42 2022] POST /identity/v3/users => generated 291 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:42.889494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:42.921990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea5ed8-42d9-4fee-ba35-8d67984c006c service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:42.944865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:42.945544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:42.945819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:42.946117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:42.946812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:42.948728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:42.983361 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 825/1662] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:00:42.984591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:43.007567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0342f3-a881-48e6-9d76-82771e6c76e2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:43.026680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a04e6e-5875-4f02-8e2f-5e5977d288f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:43.027515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a04e6e-5875-4f02-8e2f-5e5977d288f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:43.028042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a04e6e-5875-4f02-8e2f-5e5977d288f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:43.028441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a04e6e-5875-4f02-8e2f-5e5977d288f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:43.030366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a04e6e-5875-4f02-8e2f-5e5977d288f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:43.032998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a04e6e-5875-4f02-8e2f-5e5977d288f1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:43.062354 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 838/1663] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:00:43.068592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:43.081511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d45d4-cca5-4631-a138-5fa64522884e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:43.105064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0edc3f-0500-4ce6-9cfc-17c132b7ae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:43.105927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0edc3f-0500-4ce6-9cfc-17c132b7ae01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:43.106223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0edc3f-0500-4ce6-9cfc-17c132b7ae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:43.106501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0edc3f-0500-4ce6-9cfc-17c132b7ae01 admin admin] PATH_INFO: `/v3/projects/b7b97cb394eb4fa7b9a1009922085b7c/users/2cce6d8923924a6ca1bc30806a509c3e/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:43.137905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0edc3f-0500-4ce6-9cfc-17c132b7ae01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b97cb394eb4fa7b9a1009922085b7c, user_id=2cce6d8923924a6ca1bc30806a509c3e, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:43.141330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0edc3f-0500-4ce6-9cfc-17c132b7ae01 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:43.167452 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 826/1664] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:43 2022] PUT /identity/v3/projects/b7b97cb394eb4fa7b9a1009922085b7c/users/2cce6d8923924a6ca1bc30806a509c3e/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:43.197026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172275cf-d96d-4686-b558-2853a6280112 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:43.197836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172275cf-d96d-4686-b558-2853a6280112 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:43.198562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172275cf-d96d-4686-b558-2853a6280112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:43.258599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-172275cf-d96d-4686-b558-2853a6280112 None None] MFA Rules not processed for user `2cce6d8923924a6ca1bc30806a509c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:43.262867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-172275cf-d96d-4686-b558-2853a6280112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:43.323789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:43.467002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c3372-e8c7-4719-9fee-f9e314197ba0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:43.467795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c3372-e8c7-4719-9fee-f9e314197ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:43.467795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c3372-e8c7-4719-9fee-f9e314197ba0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:43.469465 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 827/1665] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:43.489704 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 839/1666] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:43.496426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c3372-e8c7-4719-9fee-f9e314197ba0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:43.518440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:43.542960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172275cf-d96d-4686-b558-2853a6280112 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:43.544986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-172275cf-d96d-4686-b558-2853a6280112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:43.704801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c3372-e8c7-4719-9fee-f9e314197ba0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:43.716021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea257a7-7360-4743-b97f-e0b3cbbcb481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:43.739633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea257a7-7360-4743-b97f-e0b3cbbcb481 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:43.740504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea257a7-7360-4743-b97f-e0b3cbbcb481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:43.741727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea257a7-7360-4743-b97f-e0b3cbbcb481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:43.747918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea257a7-7360-4743-b97f-e0b3cbbcb481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:43.759640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea257a7-7360-4743-b97f-e0b3cbbcb481 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:43.767797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172275cf-d96d-4686-b558-2853a6280112 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:43.780908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:43.781858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:43.782293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:43.782704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] PATH_INFO: `/v3/projects/14301d7630d542c3ba3f008820385459` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:43.785199 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14301d7630d542c3ba3f008820385459)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:43.806179 dv-cinder-master-3-smb devstack@keystone.service[81500]: WARNING py.warnings [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 25 01:00:43.806179 dv-cinder-master-3-smb devstack@keystone.service[81500]: warnings.warn(msg) Sep 25 01:00:43.806179 dv-cinder-master-3-smb devstack@keystone.service[81500]: Sep 25 01:00:43.809400 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:43.886763 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 828/1667] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:43 2022] POST /identity/v3/projects => generated 381 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:43.888190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee8ba-062a-4099-81e2-1252390ca272 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:43.899811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee8ba-062a-4099-81e2-1252390ca272 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:43.900353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee8ba-062a-4099-81e2-1252390ca272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.015048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-483ee8ba-062a-4099-81e2-1252390ca272 None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:44.017956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-483ee8ba-062a-4099-81e2-1252390ca272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:44.043904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14301d7630d542c3ba3f008820385459', 'request_id': 'req-526bacd5-f6a0-42da-9f45-6249543695e3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:44.065306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14301d7630d542c3ba3f008820385459', 'request_id': 'req-526bacd5-f6a0-42da-9f45-6249543695e3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:44.077561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14301d7630d542c3ba3f008820385459', 'request_id': 'req-526bacd5-f6a0-42da-9f45-6249543695e3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:44.152414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] The token cache is being invalidate because project 14301d7630d542c3ba3f008820385459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:44.153074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:44.156065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:44.164176 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 840/1668] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:43 2022] DELETE /identity/v3/projects/14301d7630d542c3ba3f008820385459 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:44.165740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:44.169098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:44.249965 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 829/1669] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:00:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:44.255215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:44.255704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:44.256248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.300053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:44.303346 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:44.412049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:44.451005 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 830/1670] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:44.451005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:44.453097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:44.478634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bacd5-f6a0-42da-9f45-6249543695e3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:44.498385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb1782-f47d-4255-ba4c-3409490845fa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:44.503299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:44.504422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:44.504911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:44.505574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.506777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:44.509931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c04893-a2be-4ea2-8c3f-5cc91d982cbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:44.510715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c04893-a2be-4ea2-8c3f-5cc91d982cbe service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:44.510846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:44.511146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c04893-a2be-4ea2-8c3f-5cc91d982cbe service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:44.511509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c04893-a2be-4ea2-8c3f-5cc91d982cbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.537632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c04893-a2be-4ea2-8c3f-5cc91d982cbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:44.545220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c04893-a2be-4ea2-8c3f-5cc91d982cbe service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:44.586964 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 831/1671] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:44.593765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8794e39-e8eb-4b9b-b042-b47fc429f8d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:44.597878 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 841/1672] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:44 2022] POST /identity/v3/users => generated 297 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:44.598041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8794e39-e8eb-4b9b-b042-b47fc429f8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:44.598041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8794e39-e8eb-4b9b-b042-b47fc429f8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.600321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:44.648977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae45fc6-86b3-4657-b817-377ab7e6d634 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:44.669423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:44.670519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:44.671027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:44.671533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.672859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:44.682538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:44.692161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e8794e39-e8eb-4b9b-b042-b47fc429f8d4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:44.699061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e8794e39-e8eb-4b9b-b042-b47fc429f8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:44.727082 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 842/1673] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:44.728289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:44.735790 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 832/1674] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:44.746757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8794e39-e8eb-4b9b-b042-b47fc429f8d4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:44.759832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d00aa2-258a-4ec1-a414-96d0203176c9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:44.775388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:44.783086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:44.783512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:44.783947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] PATH_INFO: `/v3/projects/8b049c173bf24d5291284b0ca6050893/users/e2ddfdbc4f5a4e869416fc3801464fef/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.785268 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8794e39-e8eb-4b9b-b042-b47fc429f8d4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:44.811243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:44.812821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:44.813396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:44.813907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:44.817790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:44.818365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:44.827694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:44.850553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b049c173bf24d5291284b0ca6050893, user_id=e2ddfdbc4f5a4e869416fc3801464fef, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:44.859620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:44.863891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:44.881978 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 833/1675] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:44.888842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:44.923774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3487-af54-49c8-898a-f42aa86b934a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:44.929223 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 843/1676] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:44 2022] PUT /identity/v3/projects/8b049c173bf24d5291284b0ca6050893/users/e2ddfdbc4f5a4e869416fc3801464fef/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:44.937038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:45.162407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa8e0a-0d71-45a3-aec9-b9ec44e35cfe admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:45.179591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:45.180746 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:45.181323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:45.182485 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:45.184161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:45.187430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:45.208731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:45.209863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:45.210443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:45.210948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:45.212209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:45.215590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:45.246618 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 844/1677] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:45.253880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:45.320487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b401f-6da7-4ca5-a6d4-fabfcafe0555 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:45.362541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52e73f-d5f3-4a69-b805-01c197d9fd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:45.363805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52e73f-d5f3-4a69-b805-01c197d9fd6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:45.364317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52e73f-d5f3-4a69-b805-01c197d9fd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:45.368787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52e73f-d5f3-4a69-b805-01c197d9fd6c admin admin] PATH_INFO: `/v3/projects/8b049c173bf24d5291284b0ca6050893/users/e2ddfdbc4f5a4e869416fc3801464fef/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:45.373770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52e73f-d5f3-4a69-b805-01c197d9fd6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b049c173bf24d5291284b0ca6050893, user_id=e2ddfdbc4f5a4e869416fc3801464fef, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:45.378443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52e73f-d5f3-4a69-b805-01c197d9fd6c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:45.388834 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 834/1678] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:44 2022] POST /identity/v3/projects => generated 379 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:45.391822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:45.411767 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 845/1679] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:45 2022] PUT /identity/v3/projects/8b049c173bf24d5291284b0ca6050893/users/e2ddfdbc4f5a4e869416fc3801464fef/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:45.432150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:45.432589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:45.434389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:45.559765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] MFA Rules not processed for user `e2ddfdbc4f5a4e869416fc3801464fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:45.566550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:45.703019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c853-cd03-49ad-b404-cbadfb0f3d9d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:45.884419 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 846/1680] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:45.908663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:45.910971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:45.940003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846926e9-519d-4e10-8c9f-99e0d8b4cf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:45.941666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846926e9-519d-4e10-8c9f-99e0d8b4cf96 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:45.942414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846926e9-519d-4e10-8c9f-99e0d8b4cf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:45.943127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846926e9-519d-4e10-8c9f-99e0d8b4cf96 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:45.945984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846926e9-519d-4e10-8c9f-99e0d8b4cf96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:45.955200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846926e9-519d-4e10-8c9f-99e0d8b4cf96 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:45.963016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:46.042377 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 835/1681] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:45 2022] POST /identity/v3/users => generated 290 bytes in 650 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:46.045132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:46.045632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:46.046081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:46.174312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] MFA Rules not processed for user `f32c9b07e6234d55b0d4846d44ee6ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:46.176630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:46.184102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af4418-7d6d-4ddc-b1b9-c6511a36dd86 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:46.196644 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:46.197214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:46.197877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:46.198177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:46.198621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:46.227521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:46.229051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:46.258654 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 847/1682] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:00:46.271801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:46.339350 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ada40-9dcf-440c-9940-2a9e7027f87f service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:46.386808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:46.387846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:46.389758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:46.390441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:46.393837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:46.397771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:46.436830 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 848/1683] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:46.441556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:46.455355 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 836/1684] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:46.471705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:46.472666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:46.525386 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedf751-c250-47a6-b92f-5b2e6d795f98 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:46.552135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:46.553965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:46.554939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:46.555423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:46.565938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:46.572946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:46.630574 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 849/1685] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:46.633042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:46.662298 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26d910-7c7a-47b9-9c3f-0e2bb462d527 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:46.679599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:46.680521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:46.680928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:46.681406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] PATH_INFO: `/v3/projects/b3e1628aa25c49399e4e9f54e809e064/users/67eb6bcffad04b4b8e5878910cf933f2/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:46.721217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94971-ba92-4b1f-9fbb-618c05f40c64 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:46.740353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e1628aa25c49399e4e9f54e809e064, user_id=67eb6bcffad04b4b8e5878910cf933f2, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:46.749626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb77f7-3531-49eb-9f69-1c8a36b5440e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:46.752149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:46.753847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb77f7-3531-49eb-9f69-1c8a36b5440e service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:46.754796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb77f7-3531-49eb-9f69-1c8a36b5440e service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:46.755605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb77f7-3531-49eb-9f69-1c8a36b5440e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:46.784437 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 850/1686] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:46 2022] PUT /identity/v3/projects/b3e1628aa25c49399e4e9f54e809e064/users/67eb6bcffad04b4b8e5878910cf933f2/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:46.803023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:46.806837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:46.814542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb77f7-3531-49eb-9f69-1c8a36b5440e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:46.819466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb77f7-3531-49eb-9f69-1c8a36b5440e service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:46.853072 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 837/1687] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:00:46.861275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963981c9-18f1-4e2d-8fdb-565d3f39af2a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:46.865162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963981c9-18f1-4e2d-8fdb-565d3f39af2a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:46.866261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963981c9-18f1-4e2d-8fdb-565d3f39af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:46.990827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-963981c9-18f1-4e2d-8fdb-565d3f39af2a None None] MFA Rules not processed for user `67eb6bcffad04b4b8e5878910cf933f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:46.995168 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-963981c9-18f1-4e2d-8fdb-565d3f39af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:47.136234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb6102-fc5b-44f0-a888-5265a96d3902 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:47.157095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:47.158553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:47.158874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:47.159167 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] PATH_INFO: `/v3/projects/b4ae3650ef5943a59c8c8485d1a9b983` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:47.161133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ae3650ef5943a59c8c8485d1a9b983)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:47.168414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:47.297396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ae3650ef5943a59c8c8485d1a9b983', 'request_id': 'req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:47.325162 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 838/1688] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:47.325162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963981c9-18f1-4e2d-8fdb-565d3f39af2a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:47.327496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ae3650ef5943a59c8c8485d1a9b983', 'request_id': 'req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:47.354240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ae3650ef5943a59c8c8485d1a9b983', 'request_id': 'req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:47.366812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:47.379999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] The token cache is being invalidate because project b4ae3650ef5943a59c8c8485d1a9b983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:47.384593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:47.388050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:47.390480 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 851/1689] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:46 2022] DELETE /identity/v3/projects/b4ae3650ef5943a59c8c8485d1a9b983 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:47.392021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:47.395829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:47.417340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:47.518204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963981c9-18f1-4e2d-8fdb-565d3f39af2a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:47.639596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678cbe-9787-47d1-b3fa-f42a6d2c6343 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:47.663796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:47.665346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:47.667881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c51d63-4504-4d15-adb3-97c283522c8c tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ddfdbc4f5a4e869416fc3801464fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b049c173bf24d5291284b0ca6050893', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:47.668880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:47.669433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c51d63-4504-4d15-adb3-97c283522c8c tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:47.669968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:47.670280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c51d63-4504-4d15-adb3-97c283522c8c tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:47.671044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c51d63-4504-4d15-adb3-97c283522c8c tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:47.672626 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:47.673207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c51d63-4504-4d15-adb3-97c283522c8c tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:47.673970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c51d63-4504-4d15-adb3-97c283522c8c tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:47.678220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c51d63-4504-4d15-adb3-97c283522c8c tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:47.683274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:47.714352 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 839/1690] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:47.720145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694b12-affd-403a-9d96-2fb53ef153e4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:47.721414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694b12-affd-403a-9d96-2fb53ef153e4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:47.722257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef694b12-affd-403a-9d96-2fb53ef153e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:47.770247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ef694b12-affd-403a-9d96-2fb53ef153e4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:47.773687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ef694b12-affd-403a-9d96-2fb53ef153e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:47.783537 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 852/1691] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:47 2022] POST /identity/v3/projects => generated 379 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:47.786831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:47.797471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:47.971162 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 840/1692] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:47.977761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694b12-affd-403a-9d96-2fb53ef153e4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:48.005272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef694b12-affd-403a-9d96-2fb53ef153e4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:48.021515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ff8aa-9dec-46f8-b2be-76a89c46abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:48.022614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8701874-765d-4e05-b292-e7921910f8e3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:48.022888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ff8aa-9dec-46f8-b2be-76a89c46abc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:48.023216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ff8aa-9dec-46f8-b2be-76a89c46abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:48.023601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ff8aa-9dec-46f8-b2be-76a89c46abc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:48.032975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ff8aa-9dec-46f8-b2be-76a89c46abc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:48.040458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ddfdbc4f5a4e869416fc3801464fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b049c173bf24d5291284b0ca6050893', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:48.042938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ff8aa-9dec-46f8-b2be-76a89c46abc3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:48.044231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:48.045472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:48.047316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:48.049406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:48.054391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:48.118720 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 841/1693] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:47 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:48.121340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871190fc-4ee1-474f-ae88-f1ec04e26315 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:48.121851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871190fc-4ee1-474f-ae88-f1ec04e26315 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:48.122354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871190fc-4ee1-474f-ae88-f1ec04e26315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:48.203672 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 853/1694] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:47 2022] POST /identity/v3/projects => generated 354 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:48.206962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:48.235502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-871190fc-4ee1-474f-ae88-f1ec04e26315 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:48.241054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-871190fc-4ee1-474f-ae88-f1ec04e26315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:48.474536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7224-568f-4ea2-8f17-cd1409422c0b tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:48.479133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:48.485704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:48.486928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:48.487703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:48.488837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:48.490179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:48.495155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:48.543804 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 854/1695] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:48.544077 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 842/1696] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:48.544173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:48.550584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871190fc-4ee1-474f-ae88-f1ec04e26315 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:48.560749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:48.603547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871190fc-4ee1-474f-ae88-f1ec04e26315 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:48.628171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:48.632820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:48.634941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:48.636064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] PATH_INFO: `/v3/projects/201f6b4dbd9f47249810cb63ff6eba19/users/e8d678e8cebd4801937935a5fc26d905/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:48.735575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201f6b4dbd9f47249810cb63ff6eba19, user_id=e8d678e8cebd4801937935a5fc26d905, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:48.745290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:48.787009 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 843/1697] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:48 2022] PUT /identity/v3/projects/201f6b4dbd9f47249810cb63ff6eba19/users/e8d678e8cebd4801937935a5fc26d905/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:48.789632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:48.854202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bde2d-d287-45c3-8d83-e2c5f8ee0e35 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.038198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fe5d7-0c31-470a-bdf5-5d005fc39a19 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.057580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:49.058927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.060510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.061023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.062429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:49.063131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:49.066809 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:49.068876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ddfdbc4f5a4e869416fc3801464fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b049c173bf24d5291284b0ca6050893', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:49.073294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.075005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.077745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.078750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:49.079148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:49.086087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:49.105283 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 844/1698] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:49.113755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:49.138573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f3dbc-9c7c-4a16-815b-2955c13a78ff admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.151005 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 855/1699] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 609 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:49.152508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:49.177002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:49.182101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.182691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.183395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.192106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:49.195912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cd49d-5b8d-4048-8fab-9b8133c06741 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.202395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:49.213977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:49.215323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.216796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.217092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.217856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:49.223089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:49.272193 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 845/1700] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:49.275200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:49.318942 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283eba9-b6a2-4b8e-a3c9-ee4e0a7d2fb8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.335161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:49.349686 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 856/1701] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:49 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:49.351611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:49.354068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.354618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.355120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] PATH_INFO: `/v3/projects/201f6b4dbd9f47249810cb63ff6eba19/users/e8d678e8cebd4801937935a5fc26d905/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.362897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=201f6b4dbd9f47249810cb63ff6eba19, user_id=e8d678e8cebd4801937935a5fc26d905, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:49.372732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:49.404435 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 846/1702] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:49 2022] PUT /identity/v3/projects/201f6b4dbd9f47249810cb63ff6eba19/users/e8d678e8cebd4801937935a5fc26d905/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:49.407255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:49.541871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232b676-64f9-4dde-b3a0-5b2dcd564ed0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.562126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c9093-deaa-4472-9d15-838e2d27b06a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.574796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24934ecc-16bf-4b07-90dc-ad14145111b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:49.576630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24934ecc-16bf-4b07-90dc-ad14145111b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.577214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24934ecc-16bf-4b07-90dc-ad14145111b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.578053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24934ecc-16bf-4b07-90dc-ad14145111b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.579429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24934ecc-16bf-4b07-90dc-ad14145111b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:49.596451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24934ecc-16bf-4b07-90dc-ad14145111b2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:49.649287 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 847/1703] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:49 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:49.651622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:49.653976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872dba4-492b-437d-92e2-1e36e061fa3e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.654574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872dba4-492b-437d-92e2-1e36e061fa3e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.654881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872dba4-492b-437d-92e2-1e36e061fa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.720087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2872dba4-492b-437d-92e2-1e36e061fa3e None None] MFA Rules not processed for user `e8d678e8cebd4801937935a5fc26d905`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:49.722235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2872dba4-492b-437d-92e2-1e36e061fa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:49.744251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2ddfdbc4f5a4e869416fc3801464fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b049c173bf24d5291284b0ca6050893', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:49.745963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:49.746725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:49.747290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] PATH_INFO: `/v3/projects/ae08d776612e4f81a3fc387ac8607e6c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:49.748069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:49.750008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ae08d776612e4f81a3fc387ac8607e6c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:49.753516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:49.877859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae08d776612e4f81a3fc387ac8607e6c', 'request_id': 'req-11e24999-478f-43f7-99e3-6f43a8799d85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:49.884098 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 848/1704] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:49.884098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872dba4-492b-437d-92e2-1e36e061fa3e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:49.905688 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae08d776612e4f81a3fc387ac8607e6c', 'request_id': 'req-11e24999-478f-43f7-99e3-6f43a8799d85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:49.917229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872dba4-492b-437d-92e2-1e36e061fa3e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:49.919767 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae08d776612e4f81a3fc387ac8607e6c', 'request_id': 'req-11e24999-478f-43f7-99e3-6f43a8799d85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:49.947485 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] The token cache is being invalidate because project ae08d776612e4f81a3fc387ac8607e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:49.948079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:49.951544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:49.955018 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 857/1705] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:49 2022] DELETE /identity/v3/projects/ae08d776612e4f81a3fc387ac8607e6c => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:49.959057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:49.966156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:50.225511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28ca26-bafb-432c-90dd-a9cfbe49c345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:50.225511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28ca26-bafb-432c-90dd-a9cfbe49c345 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:50.225898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28ca26-bafb-432c-90dd-a9cfbe49c345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:50.226188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28ca26-bafb-432c-90dd-a9cfbe49c345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:50.227160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28ca26-bafb-432c-90dd-a9cfbe49c345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:50.238679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28ca26-bafb-432c-90dd-a9cfbe49c345 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:50.298931 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 849/1706] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:50.308657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f06a7b-d5f0-4b9f-b7b0-b25d6b611058 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:50.308657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f06a7b-d5f0-4b9f-b7b0-b25d6b611058 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:50.308657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f06a7b-d5f0-4b9f-b7b0-b25d6b611058 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:50.308657 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 850/1707] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:00:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:50.308657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f06a7b-d5f0-4b9f-b7b0-b25d6b611058 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:50.323914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e24999-478f-43f7-99e3-6f43a8799d85 tempest-VolumeQuotaClassesTest-1157630014 tempest-VolumeQuotaClassesTest-1157630014-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:50.328165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e8f06a7b-d5f0-4b9f-b7b0-b25d6b611058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:50.344333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:50.345824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:50.346202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:50.346529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:50.353089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:50.388131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f06a7b-d5f0-4b9f-b7b0-b25d6b611058 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:50.409420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:50.410313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:50.410663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:50.410994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] PATH_INFO: `/v3/users/e20eebb25c5d4af1b39bee09f1bfb9c1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:50.422676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20eebb25c5d4af1b39bee09f1bfb9c1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:50.427387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:50.659350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20eebb25c5d4af1b39bee09f1bfb9c1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:50.671581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:50.677787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:50.682053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20eebb25c5d4af1b39bee09f1bfb9c1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:50.716336 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 858/1708] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:50.723168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:50.724665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20eebb25c5d4af1b39bee09f1bfb9c1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:50.725599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20eebb25c5d4af1b39bee09f1bfb9c1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:50.726833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:50.728835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fb931e0e-cf18-4c62-a9f2-e665d0ecc099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20eebb25c5d4af1b39bee09f1bfb9c1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:50.766499 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 851/1709] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:50 2022] DELETE /identity/v3/users/e20eebb25c5d4af1b39bee09f1bfb9c1 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:50.767871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:50.768165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:50.768621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:50.801777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520963b-7656-4f4e-bb57-b0bfa8933af4 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:50.837675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:50.868865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:50.875151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:50.941248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:51.167540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:51.168670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:51.169190 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:51.169692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:51.170927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:51.181802 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 852/1710] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:00:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:51.182837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:51.184088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:51.190071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:51.231951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6142e0b-72f8-47a6-bec1-6c4b706356f7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:51.239338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:51.240055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:51.240365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:51.240697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] PATH_INFO: `/v3/users/2cce6d8923924a6ca1bc30806a509c3e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:51.250510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cce6d8923924a6ca1bc30806a509c3e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:51.252588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:51.254830 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 859/1711] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 538 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:51.267188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:51.291014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:51.375102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cce6d8923924a6ca1bc30806a509c3e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:51.390963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cce6d8923924a6ca1bc30806a509c3e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:51.444695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cce6d8923924a6ca1bc30806a509c3e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:51.446813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cce6d8923924a6ca1bc30806a509c3e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:51.447610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cce6d8923924a6ca1bc30806a509c3e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:51.501036 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 853/1712] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:51 2022] DELETE /identity/v3/users/2cce6d8923924a6ca1bc30806a509c3e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:51.507678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:51.510002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:51.560828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f591a-c5ae-4fc6-a3bb-35660edd1887 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:51.718124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf504c68-32e9-42b1-a411-dd5d2bc73188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:51.722647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf504c68-32e9-42b1-a411-dd5d2bc73188 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:51.723220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf504c68-32e9-42b1-a411-dd5d2bc73188 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:51.724555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf504c68-32e9-42b1-a411-dd5d2bc73188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:51.729456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab7003-0631-4ada-9361-ca9ca0eab931 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:51.735566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cf504c68-32e9-42b1-a411-dd5d2bc73188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:51.745137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:51.746404 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:51.746908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:51.747384 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] PATH_INFO: `/v3/projects/b3fabf509e3f4d78b2aff11bef5ea9ba/users/32fb0132312d46bea406c0e3709f683f/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:51.782640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fabf509e3f4d78b2aff11bef5ea9ba, user_id=32fb0132312d46bea406c0e3709f683f, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:51.803424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:51.845394 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 854/1713] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:51 2022] PUT /identity/v3/projects/b3fabf509e3f4d78b2aff11bef5ea9ba/users/32fb0132312d46bea406c0e3709f683f/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:51.851442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:51.867240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:51.899335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:52.041165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf504c68-32e9-42b1-a411-dd5d2bc73188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:52.052668 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf504c68-32e9-42b1-a411-dd5d2bc73188 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:52.112955 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 860/1714] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 857 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:52.127692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:52.131055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:52.132292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:52.148432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cf9ef-1bcc-4ac0-b686-7a516fa08fb0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:52.177586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dcd57-4448-4f77-9617-a27bfdcc5e3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:52.178241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dcd57-4448-4f77-9617-a27bfdcc5e3d service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:52.178524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dcd57-4448-4f77-9617-a27bfdcc5e3d service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:52.178800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dcd57-4448-4f77-9617-a27bfdcc5e3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:52.191552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a20dcd57-4448-4f77-9617-a27bfdcc5e3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:52.198570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:52.207113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:52.227770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dcd57-4448-4f77-9617-a27bfdcc5e3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:52.230271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dcd57-4448-4f77-9617-a27bfdcc5e3d service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:52.259656 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 855/1715] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:52.262338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:52.265077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd295ec-a44d-4c46-8e24-b49ba36cefff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:52.265373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd295ec-a44d-4c46-8e24-b49ba36cefff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:52.265711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd295ec-a44d-4c46-8e24-b49ba36cefff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:52.408630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2bd295ec-a44d-4c46-8e24-b49ba36cefff None None] MFA Rules not processed for user `32fb0132312d46bea406c0e3709f683f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:52.410352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2bd295ec-a44d-4c46-8e24-b49ba36cefff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:52.480503 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 861/1716] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:52.482118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:52.484305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:52.527764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b8c6c-507d-421c-a056-8d3f82bf58cf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:52.543972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:52.544628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:52.544912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:52.545213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] PATH_INFO: `/v3/projects/6535f45ac68f4051973150a563e8db0d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:52.546819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6535f45ac68f4051973150a563e8db0d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:52.548869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:52.655200 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 856/1717] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:52.661085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:52.661771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:52.662811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:52.715025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6535f45ac68f4051973150a563e8db0d', 'request_id': 'req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:52.716717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] MFA Rules not processed for user `ef66ca98e383463994017e057c499c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:52.718753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:52.748698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6535f45ac68f4051973150a563e8db0d', 'request_id': 'req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:52.762832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6535f45ac68f4051973150a563e8db0d', 'request_id': 'req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:52.811608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] The token cache is being invalidate because project 6535f45ac68f4051973150a563e8db0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:52.815317 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:52.817247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:52.824153 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 862/1718] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:52 2022] DELETE /identity/v3/projects/6535f45ac68f4051973150a563e8db0d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:52.825577 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:52.833055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:52.876235 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 857/1719] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:52.876235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:52.879206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:52.946307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:53.035922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6682bd-b39b-48e9-8681-15db3b4b9ec9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:53.056064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:53.056925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:53.057555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:53.059629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:53.067871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:53.145160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bf126-ff94-4c66-9eda-afbe41125a9c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:53.166673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227bf42-40f7-4740-a065-6f71ad872a6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:53.168423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227bf42-40f7-4740-a065-6f71ad872a6d service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:53.168962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227bf42-40f7-4740-a065-6f71ad872a6d service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:53.169427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227bf42-40f7-4740-a065-6f71ad872a6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:53.175080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e227bf42-40f7-4740-a065-6f71ad872a6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:53.237622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:53.273142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:53.279150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:53.318602 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 863/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:53.320160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:53.325060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:53.362075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fde41b-496d-4141-af34-c476cf2e2a73 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:53.368655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:53.369398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:53.369682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:53.370008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] PATH_INFO: `/v3/users/acf570b732d045ad86af4700b999a3bb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:53.371712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf570b732d045ad86af4700b999a3bb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:53.379912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:53.420959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227bf42-40f7-4740-a065-6f71ad872a6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:53.432628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227bf42-40f7-4740-a065-6f71ad872a6d service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:53.489197 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 858/1721] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:53.493200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:53.496193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:53.496534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:53.520988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf570b732d045ad86af4700b999a3bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:53.520988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf570b732d045ad86af4700b999a3bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:53.550383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf570b732d045ad86af4700b999a3bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:53.567055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] MFA Rules not processed for user `ef66ca98e383463994017e057c499c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:53.568265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:53.588015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf570b732d045ad86af4700b999a3bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:53.588015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf570b732d045ad86af4700b999a3bb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:53.601766 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 859/1722] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:53.614156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:53.614156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:53.651650 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 864/1723] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:53 2022] DELETE /identity/v3/users/acf570b732d045ad86af4700b999a3bb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:53.683556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064559cb-3268-405f-a044-0ab70f9de3f4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:53.974656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c04f-5dc4-46af-8bef-038efb612d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:53.974656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c04f-5dc4-46af-8bef-038efb612d11 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:53.975775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c04f-5dc4-46af-8bef-038efb612d11 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:53.975775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c04f-5dc4-46af-8bef-038efb612d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:53.985482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a9c3c04f-5dc4-46af-8bef-038efb612d11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:54.059667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:54.073880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:54.148945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:54.206703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3c04f-5dc4-46af-8bef-038efb612d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:54.208628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3c04f-5dc4-46af-8bef-038efb612d11 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:54.254550 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 860/1724] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:00:54.330938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a79e31-c787-4325-880d-1f50c9f81bad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:54.343496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:54.344209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:54.344516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:54.344825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] PATH_INFO: `/v3/users/32fb0132312d46bea406c0e3709f683f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:54.350131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fb0132312d46bea406c0e3709f683f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:54.352285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:54.527215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fb0132312d46bea406c0e3709f683f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:54.537319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fb0132312d46bea406c0e3709f683f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:54.557529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fb0132312d46bea406c0e3709f683f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:54.617420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:54.667076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fb0132312d46bea406c0e3709f683f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:54.668690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fb0132312d46bea406c0e3709f683f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:54.756469 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 865/1725] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:54 2022] DELETE /identity/v3/users/32fb0132312d46bea406c0e3709f683f => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:00:54.759954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:54.760492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:54.760999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:54.834702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:54.845505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:55.086929 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 861/1726] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:55.265544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:55.273875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:55.523310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b242868-43b6-4aa6-9f8f-75d5fe28aa75 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:55.545651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:55.546964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:55.547460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:55.547950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:55.591743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:55.594731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:55.622448 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 866/1727] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:56.161403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:56.199621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:56.324605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:56.327588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:56.370045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca99ff1-174c-47b3-b585-56383dd02d44 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:56.391692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09514111-583c-45ed-80c3-6f37de1a9b18 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:56.395352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:56.396526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:56.397023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:56.397459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] PATH_INFO: `/v3/projects/68a28b8b67e34eb08d2ace41e1a947d9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:56.403007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68a28b8b67e34eb08d2ace41e1a947d9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:56.406657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:56.420018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:56.421239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:56.422522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:56.422522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] PATH_INFO: `/v3/projects/b7b97cb394eb4fa7b9a1009922085b7c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:56.424650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b97cb394eb4fa7b9a1009922085b7c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:56.434593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:56.621596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68a28b8b67e34eb08d2ace41e1a947d9', 'request_id': 'req-1aedf2fb-2945-46f3-9ede-511836d5af96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.662765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68a28b8b67e34eb08d2ace41e1a947d9', 'request_id': 'req-1aedf2fb-2945-46f3-9ede-511836d5af96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.712669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b97cb394eb4fa7b9a1009922085b7c', 'request_id': 'req-f55b3c87-5d23-42cf-9044-09268993f1fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.729579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68a28b8b67e34eb08d2ace41e1a947d9', 'request_id': 'req-1aedf2fb-2945-46f3-9ede-511836d5af96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.749045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b97cb394eb4fa7b9a1009922085b7c', 'request_id': 'req-f55b3c87-5d23-42cf-9044-09268993f1fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.760260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b97cb394eb4fa7b9a1009922085b7c', 'request_id': 'req-f55b3c87-5d23-42cf-9044-09268993f1fc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.790613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] The token cache is being invalidate because project 68a28b8b67e34eb08d2ace41e1a947d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:56.791079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.795080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1aedf2fb-2945-46f3-9ede-511836d5af96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.796966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] The token cache is being invalidate because project b7b97cb394eb4fa7b9a1009922085b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:56.800428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.801619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:56.807072 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 862/1729] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:56 2022] DELETE /identity/v3/projects/68a28b8b67e34eb08d2ace41e1a947d9 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:56.808971 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 867/1729] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:56 2022] DELETE /identity/v3/projects/b7b97cb394eb4fa7b9a1009922085b7c => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:56.814655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:56.815670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aa1dc-4276-4a70-9941-293b0ccbde43 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:56.816165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aa1dc-4276-4a70-9941-293b0ccbde43 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:56.816588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:56.824620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aa1dc-4276-4a70-9941-293b0ccbde43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:56.907099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-392aa1dc-4276-4a70-9941-293b0ccbde43 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:56.908767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-392aa1dc-4276-4a70-9941-293b0ccbde43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:57.081122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b3c87-5d23-42cf-9044-09268993f1fc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:57.100327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:57.102520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:57.102903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:57.103814 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] PATH_INFO: `/v3/users/e2ddfdbc4f5a4e869416fc3801464fef` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:57.106065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ddfdbc4f5a4e869416fc3801464fef)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:57.115961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:57.149000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:57.206239 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 863/1730] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:57.221801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a8a1d-4f45-4065-b4df-f6d234caf56c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:57.222439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a8a1d-4f45-4065-b4df-f6d234caf56c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:57.222961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a8a1d-4f45-4065-b4df-f6d234caf56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:57.266822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dd4a8a1d-4f45-4065-b4df-f6d234caf56c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:57.267143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ddfdbc4f5a4e869416fc3801464fef'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:57.269057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ddfdbc4f5a4e869416fc3801464fef'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:57.269392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dd4a8a1d-4f45-4065-b4df-f6d234caf56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:57.292117 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 864/1731] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:57.299831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ddfdbc4f5a4e869416fc3801464fef'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:57.307639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a8a1d-4f45-4065-b4df-f6d234caf56c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:57.317565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:57.344198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a8a1d-4f45-4065-b4df-f6d234caf56c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:57.359073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cb51a-d5d7-4fb3-933f-ce8b901bcb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:57.360316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb51a-d5d7-4fb3-933f-ce8b901bcb7d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:57.362676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb51a-d5d7-4fb3-933f-ce8b901bcb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:57.366223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb51a-d5d7-4fb3-933f-ce8b901bcb7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:57.368352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ddfdbc4f5a4e869416fc3801464fef'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:57.369228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ddfdbc4f5a4e869416fc3801464fef'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:57.378037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cb51a-d5d7-4fb3-933f-ce8b901bcb7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:57.378037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cb51a-d5d7-4fb3-933f-ce8b901bcb7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:57.378037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cb51a-d5d7-4fb3-933f-ce8b901bcb7d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:57.400681 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 868/1732] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:00:56 2022] DELETE /identity/v3/users/e2ddfdbc4f5a4e869416fc3801464fef => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:57.421403 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 865/1733] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:00:57.422816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:57.568359 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d697-0795-4b90-9bba-c0ccf4b5e76d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:57.569723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d697-0795-4b90-9bba-c0ccf4b5e76d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:57.573156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d697-0795-4b90-9bba-c0ccf4b5e76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:57.625080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c055d697-0795-4b90-9bba-c0ccf4b5e76d None None] MFA Rules not processed for user `f32c9b07e6234d55b0d4846d44ee6ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:57.632679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c055d697-0795-4b90-9bba-c0ccf4b5e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:57.682200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241301b0-cc8a-48d3-9732-73f359dc056b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:57.705838 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab8b36-42f5-4c51-a843-bf9242261b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:57.709893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab8b36-42f5-4c51-a843-bf9242261b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:57.714544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab8b36-42f5-4c51-a843-bf9242261b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:57.715985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab8b36-42f5-4c51-a843-bf9242261b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:57.716598 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab8b36-42f5-4c51-a843-bf9242261b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:57.728008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab8b36-42f5-4c51-a843-bf9242261b22 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:57.836703 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 869/1734] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:57 2022] POST /identity/v3/projects => generated 369 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:00:57.838526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8c036-116f-442a-a053-ddb10634b29f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:57.839984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8c036-116f-442a-a053-ddb10634b29f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:57.840584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8c036-116f-442a-a053-ddb10634b29f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:57.855161 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 870/1735] 10.222.0.26 () {54 vars in 912 bytes} [Sun Sep 25 01:00:57 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:57.856526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8c036-116f-442a-a053-ddb10634b29f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:57.868182 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 866/1736] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:57.868182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d821c1-d76c-406e-a880-45493f55086f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:57.868182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d821c1-d76c-406e-a880-45493f55086f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:57.868182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d821c1-d76c-406e-a880-45493f55086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.000560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-23d821c1-d76c-406e-a880-45493f55086f None None] MFA Rules not processed for user `8427b2d4e8c040d0a1b00e25e3b58e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:58.002009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23d821c1-d76c-406e-a880-45493f55086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:58.115807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8c036-116f-442a-a053-ddb10634b29f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:58.153987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:58.155085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.155797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.156440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.157511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:58.167302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:58.177994 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 867/1737] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:58.180017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645d9a-c7f3-445b-b290-8e82964c1164 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.181285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645d9a-c7f3-445b-b290-8e82964c1164 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.181584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645d9a-c7f3-445b-b290-8e82964c1164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.265543 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 871/1738] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:57 2022] POST /identity/v3/users => generated 285 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:58.268368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:58.283278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:58.304866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adc285-9f0c-4067-99de-fe8a16eabaad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:58.309510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-60645d9a-c7f3-445b-b290-8e82964c1164 None None] MFA Rules not processed for user `ca20e105b76149dd99e4aa64c62eb483`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:58.324871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:58.329735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.330469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.332757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.334691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60645d9a-c7f3-445b-b290-8e82964c1164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:58.336829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:58.343789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:58.381296 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 872/1739] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:58.384432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:58.398839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:58.457899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831a891-8dd6-4c93-8e48-1ddda6e715bc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:58.486427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:58.489148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.491374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.492139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.517744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:58.558077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:58.607387 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 873/1740] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:00:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:00:58.609730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:58.621040 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 868/1741] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:58.624482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.624741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.625057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.652642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9f7ff-89ad-4fdb-90f3-c3abcfb34df0 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:58.670818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:58.671547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.671890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.672219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.676776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:58.679285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:58.701034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] MFA Rules not processed for user `ca20e105b76149dd99e4aa64c62eb483`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:58.702262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:58.724344 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 869/1742] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:00:58.724691 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 874/1743] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:00:58.724819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:58.726292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:58.727018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:58.753374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f8f737-d87e-4dca-acd4-02e8693275d7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:58.765804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:58.766373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.766661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.766939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] PATH_INFO: `/v3/projects/8b049c173bf24d5291284b0ca6050893` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.769278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b049c173bf24d5291284b0ca6050893)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:58.770132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815273e-27e4-4b02-a325-991c4fcf5381 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:58.773874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:58.787306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc6dfd-0acc-4a51-9e97-d92f7ced79d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:58.788032 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc6dfd-0acc-4a51-9e97-d92f7ced79d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.788351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc6dfd-0acc-4a51-9e97-d92f7ced79d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.788733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc6dfd-0acc-4a51-9e97-d92f7ced79d2 admin admin] PATH_INFO: `/v3/projects/00fbc9531ae14e3d8634b27c5745dedb/users/87066b5eec9a45bbb0c9c7aa68650867/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.798913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:58.868716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc6dfd-0acc-4a51-9e97-d92f7ced79d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fbc9531ae14e3d8634b27c5745dedb, user_id=87066b5eec9a45bbb0c9c7aa68650867, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:58.871962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc6dfd-0acc-4a51-9e97-d92f7ced79d2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:58.932964 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 875/1744] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:00:58 2022] PUT /identity/v3/projects/00fbc9531ae14e3d8634b27c5745dedb/users/87066b5eec9a45bbb0c9c7aa68650867/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:58.955671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33234cf-c013-4d41-8a94-dee90f55d6eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:58.957174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33234cf-c013-4d41-8a94-dee90f55d6eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:58.958069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33234cf-c013-4d41-8a94-dee90f55d6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:58.969605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b049c173bf24d5291284b0ca6050893', 'request_id': 'req-2c034dca-5017-4d74-a0fa-e8660083c0c9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:58.997802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b049c173bf24d5291284b0ca6050893', 'request_id': 'req-2c034dca-5017-4d74-a0fa-e8660083c0c9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:59.025030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b049c173bf24d5291284b0ca6050893', 'request_id': 'req-2c034dca-5017-4d74-a0fa-e8660083c0c9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:59.058266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] The token cache is being invalidate because project 8b049c173bf24d5291284b0ca6050893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:00:59.060440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:59.061484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c034dca-5017-4d74-a0fa-e8660083c0c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:00:59.072098 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 870/1745] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:58 2022] DELETE /identity/v3/projects/8b049c173bf24d5291284b0ca6050893 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:00:59.073703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c33234cf-c013-4d41-8a94-dee90f55d6eb None None] MFA Rules not processed for user `87066b5eec9a45bbb0c9c7aa68650867`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:59.075450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c33234cf-c013-4d41-8a94-dee90f55d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:59.076957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.077381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.079373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.130514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:59.136056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:59.279652 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 876/1746] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:59.286112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03112c-e8cd-4b3b-8414-c3a5f4eabbe9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.286686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03112c-e8cd-4b3b-8414-c3a5f4eabbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.287045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03112c-e8cd-4b3b-8414-c3a5f4eabbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.326389 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 871/1747] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:59.328536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:59.336255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:59.359206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dcbd9-8bc9-43a1-b631-0ec8f3456eb7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:59.365985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe9da7-8b55-4789-b8cd-bb55d0b1d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:59.365985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9da7-8b55-4789-b8cd-bb55d0b1d4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.366341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9da7-8b55-4789-b8cd-bb55d0b1d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.366859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9da7-8b55-4789-b8cd-bb55d0b1d4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.367642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe9da7-8b55-4789-b8cd-bb55d0b1d4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:59.370145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe9da7-8b55-4789-b8cd-bb55d0b1d4f1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:59.405258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9b03112c-e8cd-4b3b-8414-c3a5f4eabbe9 None None] MFA Rules not processed for user `c468aa6d1248478286fe0a8792411a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:59.410874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9b03112c-e8cd-4b3b-8414-c3a5f4eabbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:59.452907 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 872/1748] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:00:59 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:59.454691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8860fe-cc38-4ec8-8e5e-6190688f7a8a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.454932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8860fe-cc38-4ec8-8e5e-6190688f7a8a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.455229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8860fe-cc38-4ec8-8e5e-6190688f7a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.479301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4d8860fe-cc38-4ec8-8e5e-6190688f7a8a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:00:59.480812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4d8860fe-cc38-4ec8-8e5e-6190688f7a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:59.658881 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 877/1749] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:59.682162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8ba22-f2b5-4567-b79e-74bc1b64470f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.683096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8ba22-f2b5-4567-b79e-74bc1b64470f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.683618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8ba22-f2b5-4567-b79e-74bc1b64470f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.685137 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 873/1750] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:00:59.686398 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 878/1751] 10.222.0.26 () {54 vars in 912 bytes} [Sun Sep 25 01:00:59 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:00:59.687921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8ba22-f2b5-4567-b79e-74bc1b64470f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:59.691771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8860fe-cc38-4ec8-8e5e-6190688f7a8a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:59.708725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-74f8ba22-f2b5-4567-b79e-74bc1b64470f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:00:59.742471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8860fe-cc38-4ec8-8e5e-6190688f7a8a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:59.763898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8ba22-f2b5-4567-b79e-74bc1b64470f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:59.765339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:59.766282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.767714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.768254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.769515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:59.776636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:59.777927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.778774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.779650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] PATH_INFO: `/v3/projects/b3fabf509e3f4d78b2aff11bef5ea9ba` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.782260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fabf509e3f4d78b2aff11bef5ea9ba)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:59.786890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:59.787129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:00:59.836281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:59.854281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:59.855748 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 874/1752] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:00:59 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:00:59.857682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:00:59.902448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cfe28-c838-453e-80af-5cf29765935e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:00:59.923478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51827ef0-4138-483c-b10a-867cfb431a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:00:59.923938 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ef0-4138-483c-b10a-867cfb431a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:00:59.924841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ef0-4138-483c-b10a-867cfb431a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:00:59.925065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ef0-4138-483c-b10a-867cfb431a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:00:59.926196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827ef0-4138-483c-b10a-867cfb431a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:00:59.926746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827ef0-4138-483c-b10a-867cfb431a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:00:59.945885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827ef0-4138-483c-b10a-867cfb431a10 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:00.018269 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 875/1753] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:00.020799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:00.021310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:00.021810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:00.030879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fabf509e3f4d78b2aff11bef5ea9ba', 'request_id': 'req-0fafebfe-748a-405c-8916-f8bebd8a8cb1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:00.057793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fabf509e3f4d78b2aff11bef5ea9ba', 'request_id': 'req-0fafebfe-748a-405c-8916-f8bebd8a8cb1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:00.081941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fabf509e3f4d78b2aff11bef5ea9ba', 'request_id': 'req-0fafebfe-748a-405c-8916-f8bebd8a8cb1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:00.098058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] MFA Rules not processed for user `8427b2d4e8c040d0a1b00e25e3b58e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:00.100253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.111771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] The token cache is being invalidate because project b3fabf509e3f4d78b2aff11bef5ea9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:00.125283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:00.128916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:00.133725 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 879/1754] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:00:59 2022] DELETE /identity/v3/projects/b3fabf509e3f4d78b2aff11bef5ea9ba => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:00.138289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:00.144017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.234266 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 876/1755] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:00.238324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:00.240755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.301501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafebfe-748a-405c-8916-f8bebd8a8cb1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:00.318731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:00.318731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:00.319194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:00.319325 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:00.320119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:00.323242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:00.361170 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 880/1756] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:00.363629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:00.372518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.394622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3de4d-586b-4f26-9b33-da3f45a682a6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:00.401256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa1c62-248f-4411-ad8e-2d5444ba8790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:00.402152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa1c62-248f-4411-ad8e-2d5444ba8790 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:00.402553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa1c62-248f-4411-ad8e-2d5444ba8790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:00.402937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa1c62-248f-4411-ad8e-2d5444ba8790 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:00.403845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa1c62-248f-4411-ad8e-2d5444ba8790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:00.406448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa1c62-248f-4411-ad8e-2d5444ba8790 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:00.414090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0df43-6b2f-4aba-a0d4-16ff58855e1d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:00.427290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:00.428302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:00.429584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:00.431081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:00.434166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.466994 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 881/1757] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:00 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:00.469572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:00.470129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:00.470755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:00.506737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:00.509077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.590120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:00.594154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:00.613011 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 877/1758] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:00.614723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:00.616895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.702625 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 882/1759] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:00.704248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:00.705631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:00.710230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:00.771163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:00.790567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff55ada-0806-495a-977f-509facf99e51 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:00.867611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fb0bf-9aba-4712-842a-c2a1a4dce298 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:00.878676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:00.880295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:00.881966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:00.882985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:00.935246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:00.940839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:00.979141 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 883/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:00.981753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:01.003593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:01.004606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:01.005060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:01.005352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:01.013109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:01.050378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:01.055319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:01.056791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e43b3-ca69-40ec-912c-adb930f8c0fe service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:01.070250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad69b2c-9be5-4a8a-a833-d19427513000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:01.071429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad69b2c-9be5-4a8a-a833-d19427513000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:01.071912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad69b2c-9be5-4a8a-a833-d19427513000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:01.073086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad69b2c-9be5-4a8a-a833-d19427513000 admin admin] PATH_INFO: `/v3/projects/147a9d2f19034d4a99d286b30bb6f66a/users/a7dfce9bda324ccb8ef76c7b89b205e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:01.084916 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 878/1761] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:01.089584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:01.137540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda5d3d-454f-42c2-a0da-b7d2dd857dca service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:01.170859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:01.171540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:01.171823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:01.172102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:01.172810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:01.178406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad69b2c-9be5-4a8a-a833-d19427513000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147a9d2f19034d4a99d286b30bb6f66a, user_id=a7dfce9bda324ccb8ef76c7b89b205e9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:01.184216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad69b2c-9be5-4a8a-a833-d19427513000 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:01.191570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:01.254740 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 884/1762] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:00 2022] PUT /identity/v3/projects/147a9d2f19034d4a99d286b30bb6f66a/users/a7dfce9bda324ccb8ef76c7b89b205e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:01.264466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e330c-15d7-454c-9a5b-1adf605295f7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:01.264881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e330c-15d7-454c-9a5b-1adf605295f7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:01.265805 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 879/1763] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:01 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:01.266847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:01.273529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e330c-15d7-454c-9a5b-1adf605295f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:01.327539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f923539-3dbc-4761-a5c4-d70a7b7936d6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:01.412692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b90e330c-15d7-454c-9a5b-1adf605295f7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:01.419932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b90e330c-15d7-454c-9a5b-1adf605295f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:01.636629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:01.637735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:01.638239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:01.638742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:01.651579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:01.665161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:01.733442 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 880/1764] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:01.748172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:01.782197 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 885/1765] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:01.785456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e330c-15d7-454c-9a5b-1adf605295f7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:01.798787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae7ec4-8e63-4023-a1ad-c632f05227d9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:01.823619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:01.826142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:01.829357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:01.831335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:01.834794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e330c-15d7-454c-9a5b-1adf605295f7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:01.835239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:01.844351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:01.894397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:01.897484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:01.903130 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 881/1766] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:01.907334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:01.907982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] PATH_INFO: `/v3/projects/147a9d2f19034d4a99d286b30bb6f66a/users/a7dfce9bda324ccb8ef76c7b89b205e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:01.910856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=147a9d2f19034d4a99d286b30bb6f66a, user_id=a7dfce9bda324ccb8ef76c7b89b205e9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:01.913858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:01.920329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:01.967071 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 886/1767] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:01 2022] PUT /identity/v3/projects/147a9d2f19034d4a99d286b30bb6f66a/users/a7dfce9bda324ccb8ef76c7b89b205e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:01.968891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:02.011132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a73bb-75fd-42ec-ba9a-f8793000dc0e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:02.202973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:02.274297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05637aa6-4e71-44fa-9ada-af2d12714a2e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:02.298380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3597c04-ccdb-4890-91ff-b99c9ab76c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:02.299857 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3597c04-ccdb-4890-91ff-b99c9ab76c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:02.301340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3597c04-ccdb-4890-91ff-b99c9ab76c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:02.302078 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3597c04-ccdb-4890-91ff-b99c9ab76c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:02.303434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3597c04-ccdb-4890-91ff-b99c9ab76c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:02.310292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3597c04-ccdb-4890-91ff-b99c9ab76c80 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:02.319688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:02.320842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:02.322523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:02.323185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:02.333207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:02.334390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:02.338361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:02.376336 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 887/1768] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:02.378842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:02.383799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:02.384316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:02.443503 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 882/1769] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:02.443503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:02.511821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70aa6d-9153-40d9-8368-9ff358122a0f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:02.519447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] MFA Rules not processed for user `a7dfce9bda324ccb8ef76c7b89b205e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:02.521620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:02.558302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:02.562294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:02.563125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:02.564642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:02.565229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] PATH_INFO: `/v3/projects/f7c6e2ed90ae46289c03511744feda65/users/fef5280eba56421bb06f93597d9bfa1d/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:02.631305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c6e2ed90ae46289c03511744feda65, user_id=fef5280eba56421bb06f93597d9bfa1d, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:02.645753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:02.712474 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 883/1770] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:02 2022] PUT /identity/v3/projects/f7c6e2ed90ae46289c03511744feda65/users/fef5280eba56421bb06f93597d9bfa1d/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:02.712474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:02.761692 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 888/1771] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:02.763081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:02.771744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:02.970453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1612767-9a0c-42a6-97c9-b7a70614f414 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:02.996009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd3265-842c-4300-9412-2add88cc1241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:02.997551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd3265-842c-4300-9412-2add88cc1241 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:03.001330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd3265-842c-4300-9412-2add88cc1241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:03.001330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd3265-842c-4300-9412-2add88cc1241 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:03.001330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd3265-842c-4300-9412-2add88cc1241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:03.009844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd3265-842c-4300-9412-2add88cc1241 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:03.025329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae00b91-b85c-4322-9c91-b4909c7ced2e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:03.160887 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 884/1772] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:02 2022] POST /identity/v3/projects => generated 363 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:03.160887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:03.160887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:03.160887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:03.345637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] MFA Rules not processed for user `fef5280eba56421bb06f93597d9bfa1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:03.354240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:03.491083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:03.496190 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:03.498409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:03.504779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:03.511474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:03.664469 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 885/1773] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:03.665620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:03.668220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:03.748090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:03.758115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:03.823265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:03.826110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:03.857617 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 889/1774] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 1095 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:03.859786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:03.888333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309bb10-13fa-4684-ad9d-e2940169a1a4 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:03.908523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af7a7e-857d-4ec6-b184-bd40e197afe9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:03.914105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:03.914780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:03.915076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:03.915357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:04.088337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:04.090111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:04.125776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:04.126765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:04.128219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:04.128584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:04.129318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:04.135172 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 886/1775] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:04.136576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:04.149288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:04.166583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4290-1a85-4889-a96b-29bdb364d37a service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:04.183683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b2f72-9b5d-42c7-b427-9863c98125b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:04.184360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2f72-9b5d-42c7-b427-9863c98125b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:04.184702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2f72-9b5d-42c7-b427-9863c98125b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:04.184980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2f72-9b5d-42c7-b427-9863c98125b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:04.185703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b2f72-9b5d-42c7-b427-9863c98125b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:04.187745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b2f72-9b5d-42c7-b427-9863c98125b0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:04.243325 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 890/1776] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:03 2022] POST /identity/v3/users => generated 282 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:04.244412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:04.254821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:04.287770 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 887/1777] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:04 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:04.290239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c462724-7bb5-4eb6-9788-ec720dd05871 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:04.298937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c462724-7bb5-4eb6-9788-ec720dd05871 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:04.299235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c462724-7bb5-4eb6-9788-ec720dd05871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:04.309559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b1ca-c21b-4528-9c5a-dc85df3f0ab6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:04.392048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3c462724-7bb5-4eb6-9788-ec720dd05871 None None] MFA Rules not processed for user `ca20e105b76149dd99e4aa64c62eb483`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:04.399871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3c462724-7bb5-4eb6-9788-ec720dd05871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:04.581226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:04.582745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:04.585974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:04.586308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] PATH_INFO: `/v3/users/67eb6bcffad04b4b8e5878910cf933f2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:04.591606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67eb6bcffad04b4b8e5878910cf933f2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:04.601471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:04.749951 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 888/1778] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:04.757272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c462724-7bb5-4eb6-9788-ec720dd05871 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:04.763604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67eb6bcffad04b4b8e5878910cf933f2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:04.765879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67eb6bcffad04b4b8e5878910cf933f2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:04.810719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c462724-7bb5-4eb6-9788-ec720dd05871 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:04.812977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67eb6bcffad04b4b8e5878910cf933f2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:04.823021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:04.835496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:04.835913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:04.836281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:04.837136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:04.840824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:04.871806 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 889/1779] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:04.875496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:04.885822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67eb6bcffad04b4b8e5878910cf933f2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:04.886143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:04.889791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67eb6bcffad04b4b8e5878910cf933f2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:04.902787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:04.934871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78310f1-0918-4a6c-8d6a-b906d9b3a823 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:04.959711 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 891/1780] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:04 2022] DELETE /identity/v3/users/67eb6bcffad04b4b8e5878910cf933f2 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:04.959711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:04.964713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f095b-68c8-40f6-9903-d197a01d4dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:04.966144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:04.967152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f095b-68c8-40f6-9903-d197a01d4dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:04.983591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f095b-68c8-40f6-9903-d197a01d4dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:04.985059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f095b-68c8-40f6-9903-d197a01d4dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:04.987335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f095b-68c8-40f6-9903-d197a01d4dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:04.990666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f095b-68c8-40f6-9903-d197a01d4dc0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:05.099715 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 890/1781] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:04 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:05.116365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafd09-e391-4a8e-99bc-7bd3cf223ab6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:05.117110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafd09-e391-4a8e-99bc-7bd3cf223ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:05.121077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafd09-e391-4a8e-99bc-7bd3cf223ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:05.266789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5dbf4-960e-4307-abc8-b36455310061 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:05.292834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:05.293392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:05.298381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:05.298769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:05.301896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:05.312084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:05.314738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a1fafd09-e391-4a8e-99bc-7bd3cf223ab6 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:05.318244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a1fafd09-e391-4a8e-99bc-7bd3cf223ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:05.361678 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 892/1782] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:05.362721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:05.387883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617c66f-abcb-4400-ae37-0ae817dd559b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:05.396597 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 891/1783] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:05.401503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fafd09-e391-4a8e-99bc-7bd3cf223ab6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:05.406214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:05.407527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:05.408032 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:05.408536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:05.409563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:05.411605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:05.438439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fafd09-e391-4a8e-99bc-7bd3cf223ab6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:05.473133 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 893/1784] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:05.474070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:05.477860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:05.479296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:05.483178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:05.492677 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:05.493479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] PATH_INFO: `/v3/projects/8243308562f9454aaf732e8334963cbd/users/91e95f03e6a04c36bd4848cadd92d96a/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:05.574916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:05.616886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8243308562f9454aaf732e8334963cbd, user_id=91e95f03e6a04c36bd4848cadd92d96a, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:05.659355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:05.746466 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 892/1785] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:05 2022] PUT /identity/v3/projects/8243308562f9454aaf732e8334963cbd/users/91e95f03e6a04c36bd4848cadd92d96a/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:05.753943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:05.763539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69577d8f-e80a-4fc0-8f45-ad3139ce90bd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:06.099907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edd087-7b9d-4c23-81f1-8056a572b8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:06.104542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd087-7b9d-4c23-81f1-8056a572b8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:06.105521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd087-7b9d-4c23-81f1-8056a572b8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:06.106303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd087-7b9d-4c23-81f1-8056a572b8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:06.120786 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc421f-448c-485d-b8c4-0de00708ae05 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:06.137324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:06.139422 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:06.140639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:06.141540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] PATH_INFO: `/v3/projects/3650fb7529d5499ab2796a656427f33a/users/bcc0bbf4e857463d8d6caf8ca791b47d/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:06.148297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edd087-7b9d-4c23-81f1-8056a572b8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:06.163822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edd087-7b9d-4c23-81f1-8056a572b8f4 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:06.237031 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 894/1786] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:06.246128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b225221-dfab-456a-b69a-49de40ed2a57 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:06.248156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b225221-dfab-456a-b69a-49de40ed2a57 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:06.248694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b225221-dfab-456a-b69a-49de40ed2a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:06.264046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3650fb7529d5499ab2796a656427f33a, user_id=bcc0bbf4e857463d8d6caf8ca791b47d, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:06.282410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:06.355681 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 893/1787] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:05 2022] PUT /identity/v3/projects/3650fb7529d5499ab2796a656427f33a/users/bcc0bbf4e857463d8d6caf8ca791b47d/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:06.357367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:06.383511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4b225221-dfab-456a-b69a-49de40ed2a57 None None] MFA Rules not processed for user `91e95f03e6a04c36bd4848cadd92d96a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:06.391820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4b225221-dfab-456a-b69a-49de40ed2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:06.456704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:06.623593 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 895/1788] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:06.629093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b225221-dfab-456a-b69a-49de40ed2a57 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:06.734038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc95b6-b1fa-4db2-a0b4-c57c6027c843 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:06.743686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:06.751674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:06.752194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:06.752711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:06.758560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:06.763565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:06.802580 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 894/1789] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:06.809191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:06.843591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd0fbe-0e72-49ed-9384-b1d6ee6e8717 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:06.871725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db817d2a-3072-4087-be7c-d119433d596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:06.872671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db817d2a-3072-4087-be7c-d119433d596b admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:06.881065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db817d2a-3072-4087-be7c-d119433d596b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:06.881587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db817d2a-3072-4087-be7c-d119433d596b admin admin] PATH_INFO: `/v3/projects/3650fb7529d5499ab2796a656427f33a/users/bcc0bbf4e857463d8d6caf8ca791b47d/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:06.888107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db817d2a-3072-4087-be7c-d119433d596b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3650fb7529d5499ab2796a656427f33a, user_id=bcc0bbf4e857463d8d6caf8ca791b47d, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:06.901485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db817d2a-3072-4087-be7c-d119433d596b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:06.945983 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 895/1790] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:06 2022] PUT /identity/v3/projects/3650fb7529d5499ab2796a656427f33a/users/bcc0bbf4e857463d8d6caf8ca791b47d/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:06.986528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:06.986850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:06.987342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:07.016935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b225221-dfab-456a-b69a-49de40ed2a57 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:07.087488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] MFA Rules not processed for user `bcc0bbf4e857463d8d6caf8ca791b47d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:07.089558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:07.191106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:07.191952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:07.192118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:07.192504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:07.193124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:07.195953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:07.275150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:07.296716 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 896/1791] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:06 2022] POST /identity/v3/projects => generated 363 bytes in 669 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:07.297755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:07.405701 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 896/1792] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:07.441226 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:07.446870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:07.534113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:07.549337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d1820-c821-49df-874c-74ecba70b9ef admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:07.565289 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:07.566474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:07.566474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:07.566474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:07.568198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:07.573426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:07.663006 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 897/1793] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:07 2022] POST /identity/v3/users => generated 288 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:07.684196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:07.761701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728b01a-385a-4622-aa80-f2532594e6ea admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:07.803375 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:07.804344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:07.804858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:07.805315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:07.817385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:07.821092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:07.947927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a50038-37cc-48bf-9687-f35e780f525a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:07.957071 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 898/1794] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:07.958900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:08.014557 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf40e5-53b5-40f9-8de9-c226b4a43f83 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:08.037165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:08.040698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:08.041351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:08.041968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] PATH_INFO: `/v3/projects/b996b61435c74d8c9caf67118b99e971/users/4a61ebd79a524da8b1ffe650c0ecd39b/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:08.102349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b996b61435c74d8c9caf67118b99e971, user_id=4a61ebd79a524da8b1ffe650c0ecd39b, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:08.115117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:08.150616 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 899/1795] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:07 2022] PUT /identity/v3/projects/b996b61435c74d8c9caf67118b99e971/users/4a61ebd79a524da8b1ffe650c0ecd39b/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:08.164068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:08.263666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dd59f-2a4a-4295-93c5-5be91f22116d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:08.265131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dd59f-2a4a-4295-93c5-5be91f22116d service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:08.265791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dd59f-2a4a-4295-93c5-5be91f22116d service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:08.273637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dd59f-2a4a-4295-93c5-5be91f22116d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:08.293003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-789dd59f-2a4a-4295-93c5-5be91f22116d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:08.392582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b36b17-5568-4248-bd99-126ce4692872 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:08.430821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:08.432116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:08.437260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:08.437733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:08.439022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:08.448127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:08.507366 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 900/1796] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:08.513735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:08.559820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c3cc-848c-4ae6-ae81-da5ba11e8384 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:08.579096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fb3ff-e43c-4c67-93f8-5d80bd6e9d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:08.586370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fb3ff-e43c-4c67-93f8-5d80bd6e9d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:08.587530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fb3ff-e43c-4c67-93f8-5d80bd6e9d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:08.589184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fb3ff-e43c-4c67-93f8-5d80bd6e9d51 admin admin] PATH_INFO: `/v3/projects/b996b61435c74d8c9caf67118b99e971/users/4a61ebd79a524da8b1ffe650c0ecd39b/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:08.599957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fb3ff-e43c-4c67-93f8-5d80bd6e9d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b996b61435c74d8c9caf67118b99e971, user_id=4a61ebd79a524da8b1ffe650c0ecd39b, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:08.613918 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fb3ff-e43c-4c67-93f8-5d80bd6e9d51 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:08.631539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dd59f-2a4a-4295-93c5-5be91f22116d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:08.641416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dd59f-2a4a-4295-93c5-5be91f22116d service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:08.675225 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 901/1797] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:08 2022] PUT /identity/v3/projects/b996b61435c74d8c9caf67118b99e971/users/4a61ebd79a524da8b1ffe650c0ecd39b/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:08.690178 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 897/1798] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 1255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:08.701488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34c49-f1f5-4ccd-bd3f-33311b3a782d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:08.703003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34c49-f1f5-4ccd-bd3f-33311b3a782d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:08.703525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34c49-f1f5-4ccd-bd3f-33311b3a782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:08.786300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:08.786949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:08.787727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:08.924736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2cb34c49-f1f5-4ccd-bd3f-33311b3a782d None None] MFA Rules not processed for user `4a61ebd79a524da8b1ffe650c0ecd39b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:08.926726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2cb34c49-f1f5-4ccd-bd3f-33311b3a782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:09.041611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:09.044054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:09.212181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:09.261495 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 902/1799] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 561 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:09.363841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34c49-f1f5-4ccd-bd3f-33311b3a782d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:09.383821 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 898/1800] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:01:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 599 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:01:09.435451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:09.627700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb34c49-f1f5-4ccd-bd3f-33311b3a782d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:09.657871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:09.661161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:09.663089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:09.663533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:09.725462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:09.728915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:09.812089 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 903/1801] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:09.861958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:09.865257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:10.542663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb0d98-bd41-4eef-a56f-0357ca9ac007 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:10.560590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:10.561410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:10.561923 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:10.562425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:10.629619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:10.637888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:10.683236 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 899/1802] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:01:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 823 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:01:10.886791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:10.931834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48358acb-e521-43af-948c-b1dd1a19c2f5 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:10.991318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:11.004710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:11.009963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:11.017189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:11.606099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:11.618624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:11.698476 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 904/1803] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 811 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:11.943213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:11.950762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:12.516312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0a6dd-3432-4f99-83b4-8f8e8d7d83f3 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:12.547169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:12.551847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:12.552639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:12.553011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] PATH_INFO: `/v3/projects/b3e1628aa25c49399e4e9f54e809e064` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:12.564586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e1628aa25c49399e4e9f54e809e064)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:12.595186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:12.811381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:12.815555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:12.852651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29c39b-5e39-4705-a585-8ff85635a9d7 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:12.921349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:12.924668 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:12.924936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:12.925215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] PATH_INFO: `/v3/users/87066b5eec9a45bbb0c9c7aa68650867` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:12.935308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] RBAC: Authorizing `identity:delete_user(user_id=87066b5eec9a45bbb0c9c7aa68650867)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:12.942767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:12.963181 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:13.031334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e1628aa25c49399e4e9f54e809e064', 'request_id': 'req-826c28e9-bb45-4653-b466-e871bd9e6403'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.076346 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e1628aa25c49399e4e9f54e809e064', 'request_id': 'req-826c28e9-bb45-4653-b466-e871bd9e6403'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.102646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e1628aa25c49399e4e9f54e809e064', 'request_id': 'req-826c28e9-bb45-4653-b466-e871bd9e6403'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.147982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] The token cache is being invalidate because project b3e1628aa25c49399e4e9f54e809e064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:13.148880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.160490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.168159 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 900/1804] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:11 2022] DELETE /identity/v3/projects/b3e1628aa25c49399e4e9f54e809e064 => generated 0 bytes in 1227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:13.174014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:13.177226 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:13.297858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87066b5eec9a45bbb0c9c7aa68650867'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.298758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87066b5eec9a45bbb0c9c7aa68650867'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.374549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87066b5eec9a45bbb0c9c7aa68650867'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.441079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87066b5eec9a45bbb0c9c7aa68650867'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.459789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-11903883-9327-41ea-a4f5-163f4cd4e47c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87066b5eec9a45bbb0c9c7aa68650867'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:13.513617 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 905/1805] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:12 2022] DELETE /identity/v3/users/87066b5eec9a45bbb0c9c7aa68650867 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:13.516422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:13.516716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:13.519383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:13.616093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c28e9-bb45-4653-b466-e871bd9e6403 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:13.701415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:13.713915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:14.053722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:14.267904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:14.269177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:14.272601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:14.273617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] PATH_INFO: `/v3/users/e8d678e8cebd4801937935a5fc26d905` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:14.289531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d678e8cebd4801937935a5fc26d905)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:14.304718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:14.376500 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 906/1806] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 862 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:14.684803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d678e8cebd4801937935a5fc26d905'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:14.711112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d678e8cebd4801937935a5fc26d905'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:14.770333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d678e8cebd4801937935a5fc26d905'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:14.773199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d678e8cebd4801937935a5fc26d905'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:14.776155 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f57fbf6-9b68-40aa-beab-e7e6e8f9cf73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d678e8cebd4801937935a5fc26d905'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:14.809704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:14.830474 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 901/1807] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:13 2022] DELETE /identity/v3/users/e8d678e8cebd4801937935a5fc26d905 => generated 0 bytes in 1657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:14.942461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:14.946498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:15.261338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf2bb3-d146-4bda-a9ef-ec1dd48f8b5c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:15.547721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:15.548677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:15.549149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:15.549632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:15.562597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:15.818843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:15.828972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:15.896245 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 907/1808] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 954 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:16.159469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:16.159469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:16.159469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:16.290230 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] MFA Rules not processed for user `8427b2d4e8c040d0a1b00e25e3b58e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:16.297275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:16.323890 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 902/1809] 10.222.0.25 () {58 vars in 1015 bytes} [Sun Sep 25 01:01:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:16.465624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:16.467353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:16.510589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994638c9-6751-4a70-8f2a-ba64cabb5205 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:16.521159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:16.523204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:16.524963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:16.525208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:16.547028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:16.550357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:16.596637 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 908/1810] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:18.677164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:18.680932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:18.933965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eb886-2624-464e-8426-5d13501c7ff6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:18.962151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c1068-3591-4482-ace0-872ef39b9c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:18.964013 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c1068-3591-4482-ace0-872ef39b9c81 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:18.967437 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c1068-3591-4482-ace0-872ef39b9c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:18.971019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c1068-3591-4482-ace0-872ef39b9c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:18.979068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba7c1068-3591-4482-ace0-872ef39b9c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:19.233805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c1068-3591-4482-ace0-872ef39b9c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:19.235600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c1068-3591-4482-ace0-872ef39b9c81 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:19.286917 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 903/1811] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:20.196027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:20.198139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:20.267401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe5af-df37-464e-8971-4163b0f99c2a service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:20.309515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:20.313296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:20.313590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:20.314005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] PATH_INFO: `/v3/projects/201f6b4dbd9f47249810cb63ff6eba19` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:20.326455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201f6b4dbd9f47249810cb63ff6eba19)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:20.329491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:20.490047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:20.571934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201f6b4dbd9f47249810cb63ff6eba19', 'request_id': 'req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:20.599170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201f6b4dbd9f47249810cb63ff6eba19', 'request_id': 'req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:20.621261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201f6b4dbd9f47249810cb63ff6eba19', 'request_id': 'req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:20.703036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] The token cache is being invalidate because project 201f6b4dbd9f47249810cb63ff6eba19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:20.711862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:20.717151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24b77554-bc5f-49b6-bcc3-afd3ce3c77a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:20.726671 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 909/1812] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:20 2022] DELETE /identity/v3/projects/201f6b4dbd9f47249810cb63ff6eba19 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:20.733483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3548a0-40b9-4bda-95e3-57f2b0b38134 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:20.734602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3548a0-40b9-4bda-95e3-57f2b0b38134 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:20.735123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3548a0-40b9-4bda-95e3-57f2b0b38134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:20.789811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0b3548a0-40b9-4bda-95e3-57f2b0b38134 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:20.797018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0b3548a0-40b9-4bda-95e3-57f2b0b38134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:21.080311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6fd8f-a680-4adb-ac10-7961f766c91f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:21.080696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6fd8f-a680-4adb-ac10-7961f766c91f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:21.080994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6fd8f-a680-4adb-ac10-7961f766c91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:21.142318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-32a6fd8f-a680-4adb-ac10-7961f766c91f None None] MFA Rules not processed for user `c468aa6d1248478286fe0a8792411a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:21.144161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-32a6fd8f-a680-4adb-ac10-7961f766c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:21.157444 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 904/1813] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:21.170390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216aff5-899c-48a2-aa6a-f629efe53eff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:21.171001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216aff5-899c-48a2-aa6a-f629efe53eff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:21.171560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216aff5-899c-48a2-aa6a-f629efe53eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:21.239560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b216aff5-899c-48a2-aa6a-f629efe53eff None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:21.248014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b216aff5-899c-48a2-aa6a-f629efe53eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:21.290656 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 905/1814] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:21.292657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216aff5-899c-48a2-aa6a-f629efe53eff None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:21.393471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216aff5-899c-48a2-aa6a-f629efe53eff None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:21.399843 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 910/1815] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:21.438722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:21.446150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:21.448731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:21.449753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:21.457123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:21.461544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:21.466202 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:21.503911 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 906/1816] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:21.511898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6fd8f-a680-4adb-ac10-7961f766c91f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:21.562730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6fd8f-a680-4adb-ac10-7961f766c91f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:21.575887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:21.577486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:21.578332 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:21.579148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:21.580415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:21.582664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:21.670101 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 911/1817] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:21 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:21.677447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:21.702985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d30789-850b-4df5-8d6f-41782d8a8e59 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:21.743321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:22.034702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:22.038262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:22.041152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:22.047053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:22.049936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:22.073231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:22.224622 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 907/1818] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:21 2022] POST /identity/v3/users => generated 279 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:01:22.230079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:22.252572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626727f9-722f-4b55-a449-f884a178ae99 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:22.273407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:22.276128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:22.276456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:22.276694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:22.279254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:22.281828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:22.312909 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 912/1819] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:22.317256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:22.366164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb29fb-9203-4308-a273-4363d13e54c8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:22.402848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44493f7-58bc-4275-806e-541e44afd39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:22.403552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44493f7-58bc-4275-806e-541e44afd39c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:22.403916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44493f7-58bc-4275-806e-541e44afd39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:22.406355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44493f7-58bc-4275-806e-541e44afd39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:22.408731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44493f7-58bc-4275-806e-541e44afd39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:22.411993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44493f7-58bc-4275-806e-541e44afd39c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:22.468867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:22.476879 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 908/1820] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:22.488330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154624ef-be2e-4915-a728-91571bec5013 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:22.520223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:22.523780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:22.524623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:22.526327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] PATH_INFO: `/v3/projects/ab16dd3fec4e4154885efb716dfb2cd5/users/d1a4b719aa2a46c5882a4e8094e22103/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:22.622471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab16dd3fec4e4154885efb716dfb2cd5, user_id=d1a4b719aa2a46c5882a4e8094e22103, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:22.631245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:22.683349 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 913/1821] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:22 2022] PUT /identity/v3/projects/ab16dd3fec4e4154885efb716dfb2cd5/users/d1a4b719aa2a46c5882a4e8094e22103/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:22.703785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb05ec5-09c1-4e2b-aafe-33fb3dbfda10 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:22.704121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb05ec5-09c1-4e2b-aafe-33fb3dbfda10 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:22.704549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb05ec5-09c1-4e2b-aafe-33fb3dbfda10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:22.711196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:22.713255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:22.839339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5bb05ec5-09c1-4e2b-aafe-33fb3dbfda10 None None] MFA Rules not processed for user `d1a4b719aa2a46c5882a4e8094e22103`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:22.841657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5bb05ec5-09c1-4e2b-aafe-33fb3dbfda10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:22.971182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:22.979423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6a4a-6045-4bc9-8b6e-666baa08f656 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:22.998014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:22.999139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:22.999780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:23.000261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] PATH_INFO: `/v3/projects/00fbc9531ae14e3d8634b27c5745dedb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:23.001791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fbc9531ae14e3d8634b27c5745dedb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:23.004353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:23.154021 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 909/1822] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:23.175089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb05ec5-09c1-4e2b-aafe-33fb3dbfda10 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:23.218805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb05ec5-09c1-4e2b-aafe-33fb3dbfda10 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:23.222344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fbc9531ae14e3d8634b27c5745dedb', 'request_id': 'req-b93ecb9b-5a40-4714-b96a-027baf742572'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:23.250152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fbc9531ae14e3d8634b27c5745dedb', 'request_id': 'req-b93ecb9b-5a40-4714-b96a-027baf742572'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:23.262885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fbc9531ae14e3d8634b27c5745dedb', 'request_id': 'req-b93ecb9b-5a40-4714-b96a-027baf742572'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:23.294696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] The token cache is being invalidate because project 00fbc9531ae14e3d8634b27c5745dedb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:23.295320 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:23.299358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:23.302778 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 914/1823] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:22 2022] DELETE /identity/v3/projects/00fbc9531ae14e3d8634b27c5745dedb => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:23.308370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:23.310316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:23.579805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ecb9b-5a40-4714-b96a-027baf742572 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:23.603502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:23.604235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:23.611801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:23.611801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] PATH_INFO: `/v3/users/a7dfce9bda324ccb8ef76c7b89b205e9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:23.613353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7dfce9bda324ccb8ef76c7b89b205e9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:23.628199 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:23.630611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999717e6-ceb0-4724-b67f-045d8be70898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:23.642353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999717e6-ceb0-4724-b67f-045d8be70898 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:23.642353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999717e6-ceb0-4724-b67f-045d8be70898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:23.642789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999717e6-ceb0-4724-b67f-045d8be70898 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:23.654692 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999717e6-ceb0-4724-b67f-045d8be70898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:23.670450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999717e6-ceb0-4724-b67f-045d8be70898 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:23.871332 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 910/1824] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:23 2022] POST /identity/v3/projects => generated 355 bytes in 712 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:23.873774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5f077-a3fe-47fd-b5ce-4627e5d8db4f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:23.874563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5f077-a3fe-47fd-b5ce-4627e5d8db4f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:23.874903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5f077-a3fe-47fd-b5ce-4627e5d8db4f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:23.877741 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 911/1825] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:01:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:01:23.879289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5f077-a3fe-47fd-b5ce-4627e5d8db4f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:23.887865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9fe5f077-a3fe-47fd-b5ce-4627e5d8db4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:23.970596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:23.985606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7dfce9bda324ccb8ef76c7b89b205e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:23.988248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7dfce9bda324ccb8ef76c7b89b205e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.020839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7dfce9bda324ccb8ef76c7b89b205e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.077966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7dfce9bda324ccb8ef76c7b89b205e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.080362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7dfce9bda324ccb8ef76c7b89b205e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.119609 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 915/1826] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:23 2022] DELETE /identity/v3/users/a7dfce9bda324ccb8ef76c7b89b205e9 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:24.122587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:24.124621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:24.146555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:24.249429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5f077-a3fe-47fd-b5ce-4627e5d8db4f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:24.372194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1390-6e87-4860-80db-0e87f4de9da1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:24.391676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ee962-5cd2-4498-9db9-8e260003c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:24.392867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ee962-5cd2-4498-9db9-8e260003c729 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:24.393760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ee962-5cd2-4498-9db9-8e260003c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:24.394161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ee962-5cd2-4498-9db9-8e260003c729 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:24.395193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ee962-5cd2-4498-9db9-8e260003c729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:24.399725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ee962-5cd2-4498-9db9-8e260003c729 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:24.487846 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 916/1827] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:24 2022] POST /identity/v3/users => generated 284 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:24.492282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:24.493774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de50440-3ad5-41b8-8e6d-5426b6e3cd54 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:24.505761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de50440-3ad5-41b8-8e6d-5426b6e3cd54 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:24.505761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de50440-3ad5-41b8-8e6d-5426b6e3cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:24.507654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:24.512175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:24.512984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] PATH_INFO: `/v3/users/91e95f03e6a04c36bd4848cadd92d96a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:24.518570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e95f03e6a04c36bd4848cadd92d96a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:24.529361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:24.593090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1de50440-3ad5-41b8-8e6d-5426b6e3cd54 None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:24.595511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1de50440-3ad5-41b8-8e6d-5426b6e3cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:24.738737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e95f03e6a04c36bd4848cadd92d96a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.756575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e95f03e6a04c36bd4848cadd92d96a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.794149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e95f03e6a04c36bd4848cadd92d96a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.795900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e95f03e6a04c36bd4848cadd92d96a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.798335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fa1cbd32-c7e0-425a-a322-99e673e8b870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e95f03e6a04c36bd4848cadd92d96a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:24.819596 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 917/1828] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:01:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:24.822968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de50440-3ad5-41b8-8e6d-5426b6e3cd54 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:24.867520 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 912/1829] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:23 2022] DELETE /identity/v3/users/91e95f03e6a04c36bd4848cadd92d96a => generated 0 bytes in 988 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:24.873756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:24.874817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:24.877403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:24.879329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de50440-3ad5-41b8-8e6d-5426b6e3cd54 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:24.989383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:24.993649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:25.168675 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:25.213735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:25.215070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:25.215754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:25.227727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:25.228994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:25.234509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:25.299457 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 918/1830] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:25.299763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:25.315503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:25.365556 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 913/1831] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:25.383661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1033ba-dd87-4dec-8941-865f06eafd1a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:25.404355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:25.405354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:25.406815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:25.407153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] PATH_INFO: `/v3/projects/d4e029bb3d604ca5ae03e461c4e33e63/users/cfc1ce1096f74f4398a93d105318d1be/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:25.469633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e029bb3d604ca5ae03e461c4e33e63, user_id=cfc1ce1096f74f4398a93d105318d1be, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:25.497274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:25.542611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:25.548014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:25.551222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:25.591588 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 919/1832] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:25 2022] PUT /identity/v3/projects/d4e029bb3d604ca5ae03e461c4e33e63/users/cfc1ce1096f74f4398a93d105318d1be/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:25.601520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:25.861534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cea3a4-c245-40ef-ae5b-2512d2e8c838 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:25.875203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:25.878521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:25.879237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:25.881398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:25.883465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:25.893104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:25.940360 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 920/1833] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:25.943038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:25.966757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32799e06-54f5-49f0-bd92-b74f6a826fbf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:25.977649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a7ca-0d93-4dd9-a03f-dd9a6821b196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:25.979020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a7ca-0d93-4dd9-a03f-dd9a6821b196 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:25.981099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a7ca-0d93-4dd9-a03f-dd9a6821b196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:25.981816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a7ca-0d93-4dd9-a03f-dd9a6821b196 admin admin] PATH_INFO: `/v3/projects/d4e029bb3d604ca5ae03e461c4e33e63/users/cfc1ce1096f74f4398a93d105318d1be/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:25.992613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a7ca-0d93-4dd9-a03f-dd9a6821b196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e029bb3d604ca5ae03e461c4e33e63, user_id=cfc1ce1096f74f4398a93d105318d1be, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:25.994186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01769f4e-4857-4b8e-a925-c887b311fe53 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:25.999884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a7ca-0d93-4dd9-a03f-dd9a6821b196 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:26.046018 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 921/1834] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:25 2022] PUT /identity/v3/projects/d4e029bb3d604ca5ae03e461c4e33e63/users/cfc1ce1096f74f4398a93d105318d1be/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:26.074206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:26.075569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:26.076043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:26.214842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] MFA Rules not processed for user `cfc1ce1096f74f4398a93d105318d1be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:26.218113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:26.277401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:26.278763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:26.279371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:26.279935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:26.338272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:26.356292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:26.401820 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 914/1835] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 856 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:01:26.688995 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 922/1836] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 617 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:26.705078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:26.710829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:26.847273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:26.852345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:26.952480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9e91e-95df-40c6-a7bf-3bc4e78502de service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:27.293043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:27.334842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:27.337030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:27.338249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:27.341247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:27.341753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8ac52-50d4-4b18-acc1-d407c3158302 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:27.368600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:27.374029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:27.374307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:27.374637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:27.375012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] PATH_INFO: `/v3/users/fef5280eba56421bb06f93597d9bfa1d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:27.380691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:27.381107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef5280eba56421bb06f93597d9bfa1d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:27.387454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:27.412065 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 915/1837] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:27.466791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:27.529383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:27.533607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:27.622318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef5280eba56421bb06f93597d9bfa1d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:27.629737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef5280eba56421bb06f93597d9bfa1d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:27.665098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef5280eba56421bb06f93597d9bfa1d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:27.712195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef5280eba56421bb06f93597d9bfa1d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:27.714293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef5280eba56421bb06f93597d9bfa1d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:27.808641 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 923/1838] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:26 2022] DELETE /identity/v3/users/fef5280eba56421bb06f93597d9bfa1d => generated 0 bytes in 959 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:27.808641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:27.814162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:28.027846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b1716-67c2-4513-a38a-bcdd7a43f40e service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:28.106508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80529dbf-89a2-4520-a8a2-3add44414243 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:28.117751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ac789-11a2-4521-ace8-fd9ef0f97401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:28.121129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ac789-11a2-4521-ace8-fd9ef0f97401 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:28.123470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ac789-11a2-4521-ace8-fd9ef0f97401 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:28.124045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ac789-11a2-4521-ace8-fd9ef0f97401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:28.130084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ae9ac789-11a2-4521-ace8-fd9ef0f97401 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:28.435137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b1555-aebf-4001-aa59-32282c29cc6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:28.437097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b1555-aebf-4001-aa59-32282c29cc6f service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:28.437598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b1555-aebf-4001-aa59-32282c29cc6f service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:28.438083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b1555-aebf-4001-aa59-32282c29cc6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:28.445719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c12b1555-aebf-4001-aa59-32282c29cc6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:28.467247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ac789-11a2-4521-ace8-fd9ef0f97401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:28.476561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ac789-11a2-4521-ace8-fd9ef0f97401 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:28.483526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b1555-aebf-4001-aa59-32282c29cc6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:28.487673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b1555-aebf-4001-aa59-32282c29cc6f service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:28.520289 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 924/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:28.524244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e73c4-cdf3-4ea7-9c0a-92b4a88360b1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:28.527183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e73c4-cdf3-4ea7-9c0a-92b4a88360b1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:28.527629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e73c4-cdf3-4ea7-9c0a-92b4a88360b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:28.535080 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 916/1840] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 1007 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:01:28.537214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa49d-135e-4763-bc44-c9d8174a59f5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:28.538727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa49d-135e-4763-bc44-c9d8174a59f5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:28.540057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa49d-135e-4763-bc44-c9d8174a59f5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:28.544887 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 917/1841] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:01:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:01:28.547402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa49d-135e-4763-bc44-c9d8174a59f5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:28.552763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-367fa49d-135e-4763-bc44-c9d8174a59f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:28.607634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cd3e73c4-cdf3-4ea7-9c0a-92b4a88360b1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:28.610231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cd3e73c4-cdf3-4ea7-9c0a-92b4a88360b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:28.813069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:28.906995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa49d-135e-4763-bc44-c9d8174a59f5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:28.938857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:28.941545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:28.943837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:28.945985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] PATH_INFO: `/v3/projects/147a9d2f19034d4a99d286b30bb6f66a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:28.953789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=147a9d2f19034d4a99d286b30bb6f66a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:28.959611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:28.974594 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 925/1842] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:28.984064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:28.984596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:28.985087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.059050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:29.060760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.060977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:29.078701 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 926/1843] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:01:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:29.097135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:29.099199 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.159662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147a9d2f19034d4a99d286b30bb6f66a', 'request_id': 'req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:29.180420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147a9d2f19034d4a99d286b30bb6f66a', 'request_id': 'req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:29.202609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147a9d2f19034d4a99d286b30bb6f66a', 'request_id': 'req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:29.260639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bbda4-5f01-469e-958c-9cf9cd098028 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:29.266438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] The token cache is being invalidate because project 147a9d2f19034d4a99d286b30bb6f66a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:29.267010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:29.270632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-07ea0ca0-8f21-434e-8aa0-8a35bea5bd1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:29.280537 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 918/1844] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:28 2022] DELETE /identity/v3/projects/147a9d2f19034d4a99d286b30bb6f66a => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:29.283158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:29.283680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:29.284148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.343973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:29.343973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.400076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ffed5-59a5-46ce-b1f5-ceaf17537f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:29.401042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ffed5-59a5-46ce-b1f5-ceaf17537f7c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:29.401433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ffed5-59a5-46ce-b1f5-ceaf17537f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:29.401998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ffed5-59a5-46ce-b1f5-ceaf17537f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.405019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-549ffed5-59a5-46ce-b1f5-ceaf17537f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.500473 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 919/1845] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:29.500473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:29.500473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.562499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ffed5-59a5-46ce-b1f5-ceaf17537f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:29.564487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ffed5-59a5-46ce-b1f5-ceaf17537f7c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:29.578069 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 927/1846] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:01:29.580281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cef04d-b11a-47dd-aeee-3398d17aae29 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:29.580891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cef04d-b11a-47dd-aeee-3398d17aae29 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:29.581184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cef04d-b11a-47dd-aeee-3398d17aae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.618338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-54cef04d-b11a-47dd-aeee-3398d17aae29 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:29.620698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-54cef04d-b11a-47dd-aeee-3398d17aae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.631378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a0414-fbb1-4d4c-b259-caa18de3319d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:29.636639 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 928/1847] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:29.642082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cef04d-b11a-47dd-aeee-3398d17aae29 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:29.646770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:29.647603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:29.648080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:29.648457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.650836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.661567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cef04d-b11a-47dd-aeee-3398d17aae29 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:29.670852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:29.674058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:29.674651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:29.675146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.676115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:29.676678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:29.678585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:29.681902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:29.686075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:29.700811 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 929/1848] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:29.702597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:29.705847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:29.715546 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 920/1849] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:29.716797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:29.731919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710b7f6-f426-4977-96b8-ced113a91fac service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:29.741928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:29.742672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:29.742884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:29.743158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.747394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc513f2-5d6f-471d-81bf-070209827a90 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:29.747940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:29.760829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:29.767664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:29.769421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:29.769913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:29.771460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] PATH_INFO: `/v3/projects/8243308562f9454aaf732e8334963cbd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:29.775607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8243308562f9454aaf732e8334963cbd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:29.792307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:29.851194 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 921/1850] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:29 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:29.852965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:29.857431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:29.884560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:29.938159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8243308562f9454aaf732e8334963cbd', 'request_id': 'req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:29.953342 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8243308562f9454aaf732e8334963cbd', 'request_id': 'req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:29.971471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8243308562f9454aaf732e8334963cbd', 'request_id': 'req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.005915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] The token cache is being invalidate because project 8243308562f9454aaf732e8334963cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:30.007340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.008858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.011423 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 930/1851] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:29 2022] DELETE /identity/v3/projects/8243308562f9454aaf732e8334963cbd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:30.023365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:30.029701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:30.067459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2695d-edcb-47e1-8b5a-de5e29f44177 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:30.210824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:30.211585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:30.212423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:30.213049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:30.213886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:30.217219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:30.236890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fc96c-0217-4ac4-8561-e5fe44e728ed admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:30.259389 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 922/1852] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:29 2022] POST /identity/v3/users => generated 288 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:30.262066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:30.266245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:30.266909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:30.267162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:30.267485 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] PATH_INFO: `/v3/users/4a61ebd79a524da8b1ffe650c0ecd39b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:30.269684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:30.272242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a61ebd79a524da8b1ffe650c0ecd39b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:30.274499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:30.306936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a7ebe-cb94-4db3-970e-db4993277404 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:30.321214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:30.322847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:30.325221 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:30.326175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:30.329334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:30.339439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:30.376123 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 923/1853] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:30.379178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:30.409439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255547c-c295-4869-a9cb-f341704543fc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:30.419551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:30.420766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:30.421502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a61ebd79a524da8b1ffe650c0ecd39b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.422041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:30.422524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:30.423658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:30.424439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a61ebd79a524da8b1ffe650c0ecd39b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.428978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:30.443008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a61ebd79a524da8b1ffe650c0ecd39b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.461023 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 924/1854] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:30.463468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:30.467841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:30.497820 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a61ebd79a524da8b1ffe650c0ecd39b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.500145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-13f70fef-7c55-4578-b4b0-7b1715b5c471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a61ebd79a524da8b1ffe650c0ecd39b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:30.526660 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 931/1855] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:30 2022] DELETE /identity/v3/users/4a61ebd79a524da8b1ffe650c0ecd39b => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:30.561075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c61a65-c7a3-4c02-a32c-f51c768c72ff admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:30.730416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:30.731459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:30.731907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:30.732424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] PATH_INFO: `/v3/projects/db2371d19d88445f9bb447e3a6e49eae/users/ed383d624e7849f780d5b8bfeec10607/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:30.754238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2371d19d88445f9bb447e3a6e49eae, user_id=ed383d624e7849f780d5b8bfeec10607, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:30.757694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:30.775575 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 925/1856] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:30 2022] PUT /identity/v3/projects/db2371d19d88445f9bb447e3a6e49eae/users/ed383d624e7849f780d5b8bfeec10607/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:30.786535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:30.786915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:30.787106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:30.826307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] MFA Rules not processed for user `ed383d624e7849f780d5b8bfeec10607`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:30.827861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:30.843303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:30.943823 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 932/1857] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:30.954706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:30.982831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:31.007903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:31.010607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:31.091850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c579d-d6c0-4b06-b419-bef0e4923ff5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:31.097591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:31.098224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.098510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.098788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.099445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:31.102240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:31.150464 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 926/1858] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:30 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:31.158389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:31.202324 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934d5e-70db-4903-a695-17f2f9dcfe4b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:31.324073 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:31.325170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.325766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.326263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] PATH_INFO: `/v3/projects/f7c6e2ed90ae46289c03511744feda65` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.327968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c6e2ed90ae46289c03511744feda65)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:31.331522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:31.333048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894be70-c0fe-4fa1-b43e-54d65092ff3e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:31.366898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:31.368057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.368624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.369116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.370355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:31.374142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:31.407361 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 927/1859] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:31 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:31.410110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:31.413599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:31.426284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c6e2ed90ae46289c03511744feda65', 'request_id': 'req-35b73855-7e07-48e8-9248-d0582b59dcb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.432967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d7967-66fd-4403-8105-b5823379d4af admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:31.440341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c6e2ed90ae46289c03511744feda65', 'request_id': 'req-35b73855-7e07-48e8-9248-d0582b59dcb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.451270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:31.453990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c6e2ed90ae46289c03511744feda65', 'request_id': 'req-35b73855-7e07-48e8-9248-d0582b59dcb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.483420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] The token cache is being invalidate because project f7c6e2ed90ae46289c03511744feda65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:31.484017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.485310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.488057 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 933/1860] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:31 2022] DELETE /identity/v3/projects/f7c6e2ed90ae46289c03511744feda65 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:31.489674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:31.494462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:31.542239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:31.543361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.543874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.544408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] PATH_INFO: `/v3/projects/b996b61435c74d8c9caf67118b99e971` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.546696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b996b61435c74d8c9caf67118b99e971)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:31.550636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:31.583657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b73855-7e07-48e8-9248-d0582b59dcb7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:31.589988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:31.590742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.591028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.591332 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.591997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:31.594729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:31.599343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:31.615490 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 934/1861] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:31.623576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:31.626704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:31.639172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b996b61435c74d8c9caf67118b99e971', 'request_id': 'req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.653582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b996b61435c74d8c9caf67118b99e971', 'request_id': 'req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.656749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efd6b6-644e-4333-a8bf-df5aca4761c7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:31.667972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b996b61435c74d8c9caf67118b99e971', 'request_id': 'req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.688540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] The token cache is being invalidate because project b996b61435c74d8c9caf67118b99e971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:31.689196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.690307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.692797 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 928/1862] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:31 2022] DELETE /identity/v3/projects/b996b61435c74d8c9caf67118b99e971 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:31.693866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:31.696220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:31.803985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:31.804975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.805263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.806327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] PATH_INFO: `/v3/users/bcc0bbf4e857463d8d6caf8ca791b47d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.806629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b79e8-1a2f-4f7f-81ca-b67a05682ff2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:31.807678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc0bbf4e857463d8d6caf8ca791b47d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:31.811968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:31.821516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166ab2b-72b5-48a5-9ed2-9ac82ff94d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:31.823230 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166ab2b-72b5-48a5-9ed2-9ac82ff94d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.824506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166ab2b-72b5-48a5-9ed2-9ac82ff94d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.825653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166ab2b-72b5-48a5-9ed2-9ac82ff94d0e admin admin] PATH_INFO: `/v3/projects/f155ce0c2a784e8c888a595ad7456185/users/10e15a75ebc6465abf41ba3e6a1873e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.877675 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166ab2b-72b5-48a5-9ed2-9ac82ff94d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f155ce0c2a784e8c888a595ad7456185, user_id=10e15a75ebc6465abf41ba3e6a1873e9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:31.884729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166ab2b-72b5-48a5-9ed2-9ac82ff94d0e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:31.932308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc0bbf4e857463d8d6caf8ca791b47d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.936184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc0bbf4e857463d8d6caf8ca791b47d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:31.938410 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 929/1863] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:31 2022] PUT /identity/v3/projects/f155ce0c2a784e8c888a595ad7456185/users/10e15a75ebc6465abf41ba3e6a1873e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:31.946864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df4eb-2d2f-49a8-bc80-13b65a12222a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:31.947309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df4eb-2d2f-49a8-bc80-13b65a12222a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:31.948026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df4eb-2d2f-49a8-bc80-13b65a12222a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:31.966568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc0bbf4e857463d8d6caf8ca791b47d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:32.001223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-282df4eb-2d2f-49a8-bc80-13b65a12222a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:32.002708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-282df4eb-2d2f-49a8-bc80-13b65a12222a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:32.010316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc0bbf4e857463d8d6caf8ca791b47d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:32.013259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc0bbf4e857463d8d6caf8ca791b47d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:32.048266 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 935/1864] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:31 2022] DELETE /identity/v3/users/bcc0bbf4e857463d8d6caf8ca791b47d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:32.049826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:32.054957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:32.068351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:32.200204 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 930/1865] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:32.212255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e367-e3c6-42a9-be76-011308079f10 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.212667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e367-e3c6-42a9-be76-011308079f10 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.212957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e367-e3c6-42a9-be76-011308079f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.216320 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f2b07-4b57-4525-bd52-ee8ca29f3b45 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:32.224256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:32.224953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.225236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.225513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.226341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:32.230222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:32.240535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9335e367-e3c6-42a9-be76-011308079f10 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:32.242909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9335e367-e3c6-42a9-be76-011308079f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:32.255453 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 936/1866] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:32.257218 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:32.264099 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 931/1867] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:32.270678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335e367-e3c6-42a9-be76-011308079f10 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:32.286358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335e367-e3c6-42a9-be76-011308079f10 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:32.287613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41084d-b339-4365-8cf9-b3d8eb2c580a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:32.295905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:32.296877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.297342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.297882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.299084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:32.299594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:32.304586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:32.320423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447caae-59fd-4454-8365-3542e51784ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:32.322816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447caae-59fd-4454-8365-3542e51784ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.323739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447caae-59fd-4454-8365-3542e51784ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.323739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447caae-59fd-4454-8365-3542e51784ed admin admin] PATH_INFO: `/v3/projects/f155ce0c2a784e8c888a595ad7456185/users/10e15a75ebc6465abf41ba3e6a1873e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.327715 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 932/1868] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:32.329191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447caae-59fd-4454-8365-3542e51784ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f155ce0c2a784e8c888a595ad7456185, user_id=10e15a75ebc6465abf41ba3e6a1873e9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:32.332476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447caae-59fd-4454-8365-3542e51784ed admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:32.334746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:32.357679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a690402-b9c9-4f79-a594-617cbf740682 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:32.386868 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 937/1869] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:32 2022] PUT /identity/v3/projects/f155ce0c2a784e8c888a595ad7456185/users/10e15a75ebc6465abf41ba3e6a1873e9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:32.407545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81248026-b166-43eb-91e0-7600e607961b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.407545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81248026-b166-43eb-91e0-7600e607961b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.408114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81248026-b166-43eb-91e0-7600e607961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.409177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:32.483973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:32.484944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.485298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.485833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.486803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:32.489625 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:32.522372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-81248026-b166-43eb-91e0-7600e607961b None None] MFA Rules not processed for user `10e15a75ebc6465abf41ba3e6a1873e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:32.528002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81248026-b166-43eb-91e0-7600e607961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:32.553856 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 933/1870] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:32 2022] POST /identity/v3/projects => generated 379 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:01:32.561892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:32.822370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a8b17-a1d4-409b-8280-59e4d790818a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:32.830201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:32.831409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.831637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.831936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.833190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:32.837682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:32.864638 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 938/1871] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:32.878438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81248026-b166-43eb-91e0-7600e607961b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:32.881078 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81248026-b166-43eb-91e0-7600e607961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:32.915291 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 934/1872] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:32 2022] POST /identity/v3/users => generated 290 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:32.919480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:32.950089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec0237-6355-44c3-9d08-0ddf65cb8960 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:32.962611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:32.964965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:32.965460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:32.965875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:32.967043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:32.970178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:32.998803 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 935/1873] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:33.005779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:33.028688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277857-eac2-4e4f-ab4a-50b6cda14496 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:33.039711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:33.041277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:33.042067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:33.043032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:33.044570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:33.047724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:33.074276 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 936/1874] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:01:33.086225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:33.086512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81248026-b166-43eb-91e0-7600e607961b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:33.121655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46b9b1-f988-4c17-94ab-df906045f487 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:33.146273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce255a-97b5-478b-a413-f5382316392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:33.147325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce255a-97b5-478b-a413-f5382316392f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:33.147635 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce255a-97b5-478b-a413-f5382316392f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:33.148071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce255a-97b5-478b-a413-f5382316392f admin admin] PATH_INFO: `/v3/projects/77ead2a617024e96956d02a0bb347597/users/0fe88990926340a4956a9f8f85f5c066/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:33.151424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:33.199960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce255a-97b5-478b-a413-f5382316392f admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ead2a617024e96956d02a0bb347597, user_id=0fe88990926340a4956a9f8f85f5c066, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:33.205650 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce255a-97b5-478b-a413-f5382316392f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:33.220126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:33.221558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:33.222054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:33.222652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:33.228202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:33.236346 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 937/1875] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:33 2022] PUT /identity/v3/projects/77ead2a617024e96956d02a0bb347597/users/0fe88990926340a4956a9f8f85f5c066/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:33.263310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:33.263827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:33.264419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:33.375013 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] MFA Rules not processed for user `0fe88990926340a4956a9f8f85f5c066`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:33.377285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:33.438582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:33.444353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:33.469647 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 939/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:33.484998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:33.662207 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 938/1877] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:01:33.671376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:33.694379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add7f26-5138-45f8-b3b4-fea23365d277 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:33.712524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:33.714428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:33.899227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:33.899731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:33.900075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:33.900509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:33.901471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:33.903538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:33.952688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ab222-ee1d-49d1-8d5e-e4a66814bd75 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:33.964427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:33.965575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:33.966078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:33.966668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] PATH_INFO: `/v3/projects/3650fb7529d5499ab2796a656427f33a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:33.968587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3650fb7529d5499ab2796a656427f33a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:33.971466 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:33.994309 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 940/1878] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:33 2022] POST /identity/v3/projects => generated 377 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:34.001729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:34.209000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3650fb7529d5499ab2796a656427f33a', 'request_id': 'req-178aea4a-f55a-4a7c-bf7e-4611998144fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:34.230589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3650fb7529d5499ab2796a656427f33a', 'request_id': 'req-178aea4a-f55a-4a7c-bf7e-4611998144fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:34.256117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3650fb7529d5499ab2796a656427f33a', 'request_id': 'req-178aea4a-f55a-4a7c-bf7e-4611998144fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:34.312957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4efb-bd72-41b5-b0fd-e790007834f4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:34.319892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] The token cache is being invalidate because project 3650fb7529d5499ab2796a656427f33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:34.320787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:34.322654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-178aea4a-f55a-4a7c-bf7e-4611998144fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:34.330984 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 939/1879] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:33 2022] DELETE /identity/v3/projects/3650fb7529d5499ab2796a656427f33a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:34.341826 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3660c-dda1-4acb-b911-ac0158e38167 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:34.342517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3660c-dda1-4acb-b911-ac0158e38167 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:34.343117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3660c-dda1-4acb-b911-ac0158e38167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:34.346709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:34.427524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-19a3660c-dda1-4acb-b911-ac0158e38167 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:34.430217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:34.431019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:34.431380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:34.431724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:34.432524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:34.433460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19a3660c-dda1-4acb-b911-ac0158e38167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:34.435359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:34.464690 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 940/1880] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:01:34.480108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26223026-9f68-4083-ae70-1100de7246be None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:34.480792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26223026-9f68-4083-ae70-1100de7246be None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:34.481424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26223026-9f68-4083-ae70-1100de7246be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:34.494126 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 941/1881] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:34 2022] POST /identity/v3/users => generated 295 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:34.517924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:34.527088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-26223026-9f68-4083-ae70-1100de7246be None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:34.529804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-26223026-9f68-4083-ae70-1100de7246be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:34.532322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:34.571428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88446c4d-141e-4f31-bfb8-89eda1e4b66f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:34.583834 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 941/1882] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:34.588099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:34.589262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:34.589759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:34.590266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:34.592152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26223026-9f68-4083-ae70-1100de7246be None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:34.594969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:34.598133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:34.624635 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26223026-9f68-4083-ae70-1100de7246be None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:34.656792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:34.656792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:34.657148 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 942/1883] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:34.657308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:34.657308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:34.667829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:34.668663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:34.668822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:34.679704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:34.706051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a884d0-a3e0-4cdf-93d6-e981cf76e9b6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:34.727473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:34.729697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:34.731233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:34.738003 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 942/1884] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:34.739526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:34.741499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] PATH_INFO: `/v3/projects/8fb931a532534c4e85852637d4edb88a/users/1ed1458aff214b70a7e783ace1330fc0/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:34.779964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832161f5-939d-4e7a-9785-c9c8ebb02330 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:34.796319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:34.797421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:34.798765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:34.800372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb931a532534c4e85852637d4edb88a, user_id=1ed1458aff214b70a7e783ace1330fc0, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:34.802190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:34.803365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:34.813604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:34.814031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:34.858929 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 943/1885] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:34 2022] PUT /identity/v3/projects/8fb931a532534c4e85852637d4edb88a/users/1ed1458aff214b70a7e783ace1330fc0/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:34.865611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:34.946324 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 943/1886] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:34 2022] POST /identity/v3/projects => generated 361 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:34.950861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:35.038593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:35.175174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a6569-ac8d-480e-96f5-b75714f3b522 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:35.186452 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:35.189303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.189303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.189600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:35.195621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:35.201502 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3f77-ab3f-4989-ab5b-74e50c5f8863 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:35.203549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:35.234469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:35.235242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.235601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.236053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:35.236860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:35.239033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:35.245941 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 944/1887] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:01:35.251620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:35.293517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6530a1-0dcc-4c9a-b3d4-d06e62b75b41 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:35.316804 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 944/1888] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:34 2022] POST /identity/v3/users => generated 281 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:35.317215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:35.318123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:35.319392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.320179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.320519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] PATH_INFO: `/v3/projects/8fb931a532534c4e85852637d4edb88a/users/1ed1458aff214b70a7e783ace1330fc0/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:35.328141 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb931a532534c4e85852637d4edb88a, user_id=1ed1458aff214b70a7e783ace1330fc0, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:35.341194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:35.347726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dfdda-fe90-4d7a-ab47-0fb7f31df6f5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:35.357049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000243e-ebae-40f7-b510-e0c68d5381fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:35.357775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000243e-ebae-40f7-b510-e0c68d5381fd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.358217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000243e-ebae-40f7-b510-e0c68d5381fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.358503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000243e-ebae-40f7-b510-e0c68d5381fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:35.359216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000243e-ebae-40f7-b510-e0c68d5381fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:35.361588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000243e-ebae-40f7-b510-e0c68d5381fd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:35.384361 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 945/1889] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:35 2022] PUT /identity/v3/projects/8fb931a532534c4e85852637d4edb88a/users/1ed1458aff214b70a7e783ace1330fc0/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:35.390906 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 945/1890] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:35.394805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:35.400250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.400837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.401279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:35.508130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] MFA Rules not processed for user `1ed1458aff214b70a7e783ace1330fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:35.512225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:35.590641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:35.607019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca48cae-091e-463b-a67f-2ae8cd8e42cd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:35.619147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:35.620329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.621457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.621457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:35.622903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:35.633853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:35.688191 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 946/1891] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:35.721270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:35.781894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fb797-878c-4f99-a46a-41cfbc71be63 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:35.794781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a7047-b187-44c2-aa2b-83b0688f1fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:35.795945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a7047-b187-44c2-aa2b-83b0688f1fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.796517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a7047-b187-44c2-aa2b-83b0688f1fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.798202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a7047-b187-44c2-aa2b-83b0688f1fdd admin admin] PATH_INFO: `/v3/projects/2728365f05f54a58ba12c81ef2e5d381/users/24b974dd76d240f192559c75923dcc5c/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:35.826512 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 946/1892] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:35.846873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:35.852117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a7047-b187-44c2-aa2b-83b0688f1fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2728365f05f54a58ba12c81ef2e5d381, user_id=24b974dd76d240f192559c75923dcc5c, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:35.852618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:35.858151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a7047-b187-44c2-aa2b-83b0688f1fdd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:35.890886 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 947/1893] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:35 2022] PUT /identity/v3/projects/2728365f05f54a58ba12c81ef2e5d381/users/24b974dd76d240f192559c75923dcc5c/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:35.920685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:35.921597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:35.922323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:36.042428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] MFA Rules not processed for user `24b974dd76d240f192559c75923dcc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:36.043957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:36.087313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde6491-2975-41ea-bed7-4d359cd8f468 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:36.191850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:36.251255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:36.252310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:36.252814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:36.253464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:36.257862 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:36.381397 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 948/1894] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:36.460013 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:36.474789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:36.502648 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 947/1895] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:01:36.536336 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:36.539316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:36.561082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee162f2-f30b-49e5-bdca-ebfd5d62c5cd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:36.565630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d1a7b-0459-415b-8a6b-5e29916fdae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:36.566466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1a7b-0459-415b-8a6b-5e29916fdae6 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:36.566850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1a7b-0459-415b-8a6b-5e29916fdae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:36.567229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d1a7b-0459-415b-8a6b-5e29916fdae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:36.578337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1a7b-0459-415b-8a6b-5e29916fdae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:36.580216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d1a7b-0459-415b-8a6b-5e29916fdae6 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:36.593112 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 949/1896] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:36.619450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:36.749618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a144-d5ab-40c4-a116-4c1d481a82a8 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:36.767086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:36.773949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:36.773949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:36.773949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] PATH_INFO: `/v3/users/24b974dd76d240f192559c75923dcc5c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:36.776350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b974dd76d240f192559c75923dcc5c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:36.781139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:36.964832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b974dd76d240f192559c75923dcc5c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:36.991637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b974dd76d240f192559c75923dcc5c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:37.049780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b974dd76d240f192559c75923dcc5c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:37.049780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b974dd76d240f192559c75923dcc5c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:37.054086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b974dd76d240f192559c75923dcc5c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:37.101359 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 948/1897] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:36 2022] DELETE /identity/v3/users/24b974dd76d240f192559c75923dcc5c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:37.104546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:37.105095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:37.105623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:37.166212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:37.168446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:37.273796 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 950/1898] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:37.283151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:37.286002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:37.302451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:37.398721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe5a0c-72c7-4c77-9646-fdc0b7bd809a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:37.410738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d561f-21ef-4286-9c9b-64b94799c3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:37.411546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d561f-21ef-4286-9c9b-64b94799c3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:37.411870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d561f-21ef-4286-9c9b-64b94799c3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:37.412292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d561f-21ef-4286-9c9b-64b94799c3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:37.423976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d561f-21ef-4286-9c9b-64b94799c3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:37.425774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d561f-21ef-4286-9c9b-64b94799c3e2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:37.440650 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 949/1899] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:01:37.541667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:37.544426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:37.632766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:37.657776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1023b-37ae-4fa1-b75a-e83257cf50b6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:37.668133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:37.669929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:37.670373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:37.670929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:37.676863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:37.807372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12654a3d-d69f-4155-91fe-d14c7675e6d7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:37.807826 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12654a3d-d69f-4155-91fe-d14c7675e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:37.808315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12654a3d-d69f-4155-91fe-d14c7675e6d7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:37.811644 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 950/1900] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:01:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:01:37.825584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:37.825911 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:37.826294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:37.862821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:37.866387 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:37.872656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:37.875301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:37.892416 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 951/1901] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:38.064007 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 951/1902] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:01:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:01:38.080305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:38.086884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:38.126476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9955611-d8aa-4ff1-af83-d6ea0848411d service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:38.154239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:38.155228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:38.155912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:38.156417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:38.174820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:38.177471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:38.222098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:38.229163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:38.232235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975df295-bf7e-4fe0-9a85-6add0bb27937 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:38.242198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:38.243490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:38.251744 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 952/1903] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:38.253479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:38.254239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] PATH_INFO: `/v3/projects/2728365f05f54a58ba12c81ef2e5d381` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:38.258411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2728365f05f54a58ba12c81ef2e5d381)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:38.263094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:38.389447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:38.392822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:38.409488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906ef1-4633-430c-8172-8463553a3fe4 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:38.473855 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2728365f05f54a58ba12c81ef2e5d381', 'request_id': 'req-dbd45538-c87f-480e-a86f-322057ec3a13'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:38.486496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2728365f05f54a58ba12c81ef2e5d381', 'request_id': 'req-dbd45538-c87f-480e-a86f-322057ec3a13'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:38.503858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2728365f05f54a58ba12c81ef2e5d381', 'request_id': 'req-dbd45538-c87f-480e-a86f-322057ec3a13'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:38.532284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] The token cache is being invalidate because project 2728365f05f54a58ba12c81ef2e5d381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:38.532589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:38.534981 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dbd45538-c87f-480e-a86f-322057ec3a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:38.541164 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 952/1904] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:38 2022] DELETE /identity/v3/projects/2728365f05f54a58ba12c81ef2e5d381 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:38.547481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32978c-0d69-405d-adf7-d7ed077253ef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:38.548285 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32978c-0d69-405d-adf7-d7ed077253ef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:38.549907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32978c-0d69-405d-adf7-d7ed077253ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:38.556617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:38.557500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:38.557842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:38.558215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:38.581233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:38.589128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1f32978c-0d69-405d-adf7-d7ed077253ef None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:38.590949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1f32978c-0d69-405d-adf7-d7ed077253ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:38.636754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:38.778071 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 953/1905] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:38.787591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549423-e65f-4b63-bdef-a946e6cd9b06 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:38.788006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549423-e65f-4b63-bdef-a946e6cd9b06 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:38.788750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b549423-e65f-4b63-bdef-a946e6cd9b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:38.841975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:38.843482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:38.860039 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 953/1906] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:01:38.870516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3b549423-e65f-4b63-bdef-a946e6cd9b06 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:38.873477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3b549423-e65f-4b63-bdef-a946e6cd9b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:38.900123 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 954/1907] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:01:38.904746 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:38.920069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734b221-ebab-425f-bf27-33d9417ec554 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:38.927272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:38.928338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:38.928708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:38.930002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:38.932154 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:38.932874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:38.936780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:38.978414 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 954/1908] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:38.980047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b549423-e65f-4b63-bdef-a946e6cd9b06 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:39.000585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b549423-e65f-4b63-bdef-a946e6cd9b06 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:39.007695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:39.008400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:39.008696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:39.008969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:39.009610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:39.013172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:39.076445 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 955/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:38 2022] POST /identity/v3/projects => generated 367 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:39.079040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:39.117178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86fb66-99c9-43bd-b8be-24817ea22258 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:39.152633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:39.306963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:39.308014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:39.308549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:39.309028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:39.310776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:39.314936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:39.392979 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 955/1910] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:39 2022] POST /identity/v3/users => generated 284 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:39.398408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:39.412558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaa519-4e5d-4ea8-8482-df407b3dd6f7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:39.425064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:39.426006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:39.426501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:39.426820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:39.427575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:39.430701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:39.453468 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 956/1911] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:39.459586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:39.502201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8eac7-ab55-4bec-93e8-5323134d4dac admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:39.519502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17436695-4a55-4d21-8c9d-7e988371860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:39.520607 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17436695-4a55-4d21-8c9d-7e988371860e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:39.521108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17436695-4a55-4d21-8c9d-7e988371860e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:39.521601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17436695-4a55-4d21-8c9d-7e988371860e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:39.523330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17436695-4a55-4d21-8c9d-7e988371860e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:39.526592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17436695-4a55-4d21-8c9d-7e988371860e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:39.559683 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 956/1912] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:01:39.560700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:39.579988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64463ad-9207-4506-892a-2879c9a455fa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:39.589877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:39.591465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:39.592199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:39.592980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] PATH_INFO: `/v3/projects/7a7d872310ec4939b3fbe251dd5e6c24/users/1e4901b9773843cbb36d0026a4bd607c/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:39.620596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7d872310ec4939b3fbe251dd5e6c24, user_id=1e4901b9773843cbb36d0026a4bd607c, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:39.624190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:39.644487 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 957/1913] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:39 2022] PUT /identity/v3/projects/7a7d872310ec4939b3fbe251dd5e6c24/users/1e4901b9773843cbb36d0026a4bd607c/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:39.661170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:39.661616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:39.662266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:39.727075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] MFA Rules not processed for user `1e4901b9773843cbb36d0026a4bd607c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:39.729021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:39.943509 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 957/1914] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:01:39.983146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:39.996481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:40.201270 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5c1ff-0c26-481d-b9d4-a5735ccb68cb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:40.207817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ccc26-d2f9-489a-8fb5-39cdfc453790 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:40.208939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ccc26-d2f9-489a-8fb5-39cdfc453790 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:40.209507 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ccc26-d2f9-489a-8fb5-39cdfc453790 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:40.211348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ccc26-d2f9-489a-8fb5-39cdfc453790 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:40.232568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ccc26-d2f9-489a-8fb5-39cdfc453790 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:40.235335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ccc26-d2f9-489a-8fb5-39cdfc453790 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:40.262523 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 958/1915] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:43.594311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:43.599913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:43.898830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20825fb-68ca-48ac-8727-99038f7c37a6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:43.920087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:43.921273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:43.921755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:43.922233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] PATH_INFO: `/v3/users/0fe88990926340a4956a9f8f85f5c066` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:43.929937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe88990926340a4956a9f8f85f5c066)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:43.939907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:44.036009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:44.081331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe88990926340a4956a9f8f85f5c066'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:44.093620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe88990926340a4956a9f8f85f5c066'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:44.118423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe88990926340a4956a9f8f85f5c066'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:44.176178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe88990926340a4956a9f8f85f5c066'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:44.177477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe88990926340a4956a9f8f85f5c066'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:44.217798 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 958/1916] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:43 2022] DELETE /identity/v3/users/0fe88990926340a4956a9f8f85f5c066 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:44.224627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:44.225385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:44.225385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:44.268124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:44.270390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:44.361464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:44.404231 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 959/1917] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:44.513143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:44.514934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:44.812472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b690565-bd21-47fd-906b-0c0ed286854d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:44.837363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:44.842365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:44.842987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:44.843378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:44.885796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:44.896012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:44.943732 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 959/1918] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:46.435518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:46.437892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:46.454891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64ae75-472a-44d2-be00-da554a29f4cb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:46.507730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:46.509306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:46.511525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:46.512058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] PATH_INFO: `/v3/projects/77ead2a617024e96956d02a0bb347597` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:46.520641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ead2a617024e96956d02a0bb347597)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:46.528933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:46.719080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ead2a617024e96956d02a0bb347597', 'request_id': 'req-51adbf55-063d-40a7-ab42-3cd50d0176b6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:46.742584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ead2a617024e96956d02a0bb347597', 'request_id': 'req-51adbf55-063d-40a7-ab42-3cd50d0176b6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:46.785015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ead2a617024e96956d02a0bb347597', 'request_id': 'req-51adbf55-063d-40a7-ab42-3cd50d0176b6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:46.873706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] The token cache is being invalidate because project 77ead2a617024e96956d02a0bb347597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:46.874142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:46.877561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:46.884408 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 960/1919] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:46 2022] DELETE /identity/v3/projects/77ead2a617024e96956d02a0bb347597 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:46.890364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:46.920898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55709aaf-a1ab-4ac7-a85e-aa2d722dcb12 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:47.220851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:47.222030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:47.222547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:47.222980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] PATH_INFO: `/v3/users/1ed1458aff214b70a7e783ace1330fc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:47.225368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed1458aff214b70a7e783ace1330fc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:47.229494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:47.404892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed1458aff214b70a7e783ace1330fc0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:47.406370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed1458aff214b70a7e783ace1330fc0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:47.422943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:47.447153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed1458aff214b70a7e783ace1330fc0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:47.487604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed1458aff214b70a7e783ace1330fc0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:47.489581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a9ff2615-3701-4642-8088-d69609fc7078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed1458aff214b70a7e783ace1330fc0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:47.546945 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 960/1920] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:46 2022] DELETE /identity/v3/users/1ed1458aff214b70a7e783ace1330fc0 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:48.828910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:48.831730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:48.912787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adbf55-063d-40a7-ab42-3cd50d0176b6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:49.084048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:49.084810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:49.085390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:49.085902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] PATH_INFO: `/v3/projects/8fb931a532534c4e85852637d4edb88a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:49.087219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb931a532534c4e85852637d4edb88a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:49.089738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:49.241218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb931a532534c4e85852637d4edb88a', 'request_id': 'req-d1296d46-61ac-4ee0-b975-90e31a0aa81d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:49.255354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb931a532534c4e85852637d4edb88a', 'request_id': 'req-d1296d46-61ac-4ee0-b975-90e31a0aa81d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:49.267626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb931a532534c4e85852637d4edb88a', 'request_id': 'req-d1296d46-61ac-4ee0-b975-90e31a0aa81d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:49.286405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:49.308637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] The token cache is being invalidate because project 8fb931a532534c4e85852637d4edb88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:49.311428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:49.313141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1296d46-61ac-4ee0-b975-90e31a0aa81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:49.324588 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 961/1921] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:48 2022] DELETE /identity/v3/projects/8fb931a532534c4e85852637d4edb88a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:49.336963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45e4f1-af1b-4af4-9646-db7f171a8308 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:49.337802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45e4f1-af1b-4af4-9646-db7f171a8308 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:49.337979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45e4f1-af1b-4af4-9646-db7f171a8308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:49.426601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1b45e4f1-af1b-4af4-9646-db7f171a8308 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:49.449896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1b45e4f1-af1b-4af4-9646-db7f171a8308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:49.737443 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 961/1922] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:49.758322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bd0b-724f-4cbe-95dc-5f811d6fbdd7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:49.758982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bd0b-724f-4cbe-95dc-5f811d6fbdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:49.759520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bd0b-724f-4cbe-95dc-5f811d6fbdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:49.872083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-17e5bd0b-724f-4cbe-95dc-5f811d6fbdd7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:49.874968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-17e5bd0b-724f-4cbe-95dc-5f811d6fbdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:49.898044 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 962/1923] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:49.905227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45e4f1-af1b-4af4-9646-db7f171a8308 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:49.966545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45e4f1-af1b-4af4-9646-db7f171a8308 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:50.019811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:50.021801 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:50.025687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:50.028556 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:50.030343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:50.033990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:50.045885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:50.137405 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 962/1924] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:50.142563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5bd0b-724f-4cbe-95dc-5f811d6fbdd7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:50.213806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5bd0b-724f-4cbe-95dc-5f811d6fbdd7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:50.232941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:50.234423 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:50.234939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:50.235798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:50.237396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:50.243133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:50.302661 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 963/1925] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:50 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:01:50.308039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:50.333521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1eb843-0632-4a9b-b5f8-37187aa9faa0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:50.463787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:50.465143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:50.465143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:50.465658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:50.467275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:50.470007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:50.522897 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 963/1926] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:50 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:50.525257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:50.548623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001f8bd-1b26-4cea-bce0-f15d15d49575 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:50.571297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:50.572180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:50.572665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:50.573000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:50.573836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:50.577190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:50.604683 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 964/1927] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:50.605989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:50.626124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6fd18-8e33-4d68-9598-8a16f2be58f8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:50.641342 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8f1b7-68b4-49db-bc2d-c68ef57a7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:50.642106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f1b7-68b4-49db-bc2d-c68ef57a7d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:50.642490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f1b7-68b4-49db-bc2d-c68ef57a7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:50.642834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f1b7-68b4-49db-bc2d-c68ef57a7d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:50.643638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8f1b7-68b4-49db-bc2d-c68ef57a7d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:50.645971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8f1b7-68b4-49db-bc2d-c68ef57a7d7e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:50.662192 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 964/1928] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:50.668059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:50.685366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aee7b5-1cfe-4fb1-aa6a-5043d52bf9e5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:50.692903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:50.696068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:50.696068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:50.696068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] PATH_INFO: `/v3/projects/a634ca0915fc4f3b8be7f553467e9f36/users/1479d942325744e0a04b7ba6f941c62b/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:50.758036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a634ca0915fc4f3b8be7f553467e9f36, user_id=1479d942325744e0a04b7ba6f941c62b, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:50.774281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:50.812884 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 965/1929] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:50 2022] PUT /identity/v3/projects/a634ca0915fc4f3b8be7f553467e9f36/users/1479d942325744e0a04b7ba6f941c62b/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:50.825761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe00be-1acd-4113-9103-f54c292b32f4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:50.826041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe00be-1acd-4113-9103-f54c292b32f4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:50.826332 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe00be-1acd-4113-9103-f54c292b32f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:50.865480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e9fe00be-1acd-4113-9103-f54c292b32f4 None None] MFA Rules not processed for user `1479d942325744e0a04b7ba6f941c62b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:50.867471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e9fe00be-1acd-4113-9103-f54c292b32f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:50.905618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:51.007479 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 965/1930] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:51.015283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:51.197194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95ca76-fc07-45e1-9016-e44962fe9eaf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:51.205769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:51.206610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:51.207130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:51.208459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:51.208459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:51.211795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:51.280496 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 966/1931] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:51 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:51.289553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe00be-1acd-4113-9103-f54c292b32f4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:51.319118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe00be-1acd-4113-9103-f54c292b32f4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:51.539936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:51.541264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:51.541921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:51.542634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:51.543852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:51.551917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:51.625769 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 966/1932] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:51 2022] POST /identity/v3/users => generated 299 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:51.630225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:51.649984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde68e71-5b1d-4cd4-9fb4-c46f9d04865e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:51.674417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:51.675525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:51.676018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:51.676548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:51.677748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:51.682212 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:51.711113 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 967/1933] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:51.715706 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:51.734482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad53523-03d7-4fb8-854d-58cf5f311637 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:51.740341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:51.741862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:51.742328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:51.742615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] PATH_INFO: `/v3/projects/395beb87b81944e395ed04763fc6a0a7/users/f51fb49a88824a3ab6c4e6e858901bc9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:51.775044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395beb87b81944e395ed04763fc6a0a7, user_id=f51fb49a88824a3ab6c4e6e858901bc9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:51.778420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:51.803499 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 967/1934] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:51 2022] PUT /identity/v3/projects/395beb87b81944e395ed04763fc6a0a7/users/f51fb49a88824a3ab6c4e6e858901bc9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:51.805836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:51.827498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ef10a-b265-460a-8e81-39cd0b863730 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:51.939673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:51.941775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:52.072273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:52.076316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:52.076884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:52.077661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:52.079123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:52.089822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:52.135396 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 968/1935] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:52.143904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:52.191308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9092-fe7e-4f39-af80-1efa56ea7cf4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:52.193469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c7218-9499-43c0-bc37-21fc829e8a75 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:52.200257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24fbf1-b5e2-41aa-ae20-2edcfda34270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:52.200985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24fbf1-b5e2-41aa-ae20-2edcfda34270 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:52.201269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24fbf1-b5e2-41aa-ae20-2edcfda34270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:52.201545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24fbf1-b5e2-41aa-ae20-2edcfda34270 admin admin] PATH_INFO: `/v3/projects/395beb87b81944e395ed04763fc6a0a7/users/f51fb49a88824a3ab6c4e6e858901bc9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:52.206312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24fbf1-b5e2-41aa-ae20-2edcfda34270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395beb87b81944e395ed04763fc6a0a7, user_id=f51fb49a88824a3ab6c4e6e858901bc9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:52.207734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:52.213190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24fbf1-b5e2-41aa-ae20-2edcfda34270 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:52.214350 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:52.215119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:52.216106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] PATH_INFO: `/v3/users/d1a4b719aa2a46c5882a4e8094e22103` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:52.216370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:52.244434 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 969/1936] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:52 2022] PUT /identity/v3/projects/395beb87b81944e395ed04763fc6a0a7/users/f51fb49a88824a3ab6c4e6e858901bc9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:52.245181 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a4b719aa2a46c5882a4e8094e22103)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:52.245181 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:52.255337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:52.255686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:52.255959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:52.305318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] MFA Rules not processed for user `f51fb49a88824a3ab6c4e6e858901bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:52.305318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:52.328594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:52.356834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a4b719aa2a46c5882a4e8094e22103'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:52.357812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a4b719aa2a46c5882a4e8094e22103'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:52.369176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a4b719aa2a46c5882a4e8094e22103'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:52.393747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a4b719aa2a46c5882a4e8094e22103'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:52.396982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e467cdb8-0e57-4822-96f1-b1ed520f6996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a4b719aa2a46c5882a4e8094e22103'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:52.435876 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 968/1937] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:51 2022] DELETE /identity/v3/users/d1a4b719aa2a46c5882a4e8094e22103 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:52.441446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:52.442009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:52.442540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:52.480799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:52.482724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:52.602103 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 970/1938] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:52.693607 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 969/1939] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:52.701177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:52.705152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:52.721389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:52.724589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:52.963524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b15cc-75b4-4724-9c0b-dcfb36fef498 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:52.970203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:52.971752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:52.972470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:52.972903 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:52.990017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cadc10-c472-424b-acf0-63b9d339efb9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:53.000915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:53.003372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:53.031613 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 970/1940] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:53.143476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aea3d-f5c3-45a0-9544-07e274e8470b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:53.144467 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aea3d-f5c3-45a0-9544-07e274e8470b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:53.144637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aea3d-f5c3-45a0-9544-07e274e8470b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:53.144810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aea3d-f5c3-45a0-9544-07e274e8470b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:53.268906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:53.322073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aea3d-f5c3-45a0-9544-07e274e8470b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:53.327210 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aea3d-f5c3-45a0-9544-07e274e8470b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:53.369607 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 971/1941] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:53.508097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:53.510695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:53.541212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d4791-8734-413d-87aa-78b9c39afabb service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:53.547854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:53.549176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:53.549948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:53.550945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] PATH_INFO: `/v3/users/1479d942325744e0a04b7ba6f941c62b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:53.552705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1479d942325744e0a04b7ba6f941c62b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:53.555752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:53.669115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:53.671565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1479d942325744e0a04b7ba6f941c62b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:53.672723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1479d942325744e0a04b7ba6f941c62b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:53.690393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1479d942325744e0a04b7ba6f941c62b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:53.719891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1479d942325744e0a04b7ba6f941c62b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:53.729807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1479d942325744e0a04b7ba6f941c62b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:53.785181 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 971/1942] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:53 2022] DELETE /identity/v3/users/1479d942325744e0a04b7ba6f941c62b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:53.789328 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:53.789883 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:53.790381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:53.885393 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:53.887723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:53.948049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:53.950196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:54.117020 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 972/1943] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:54.149297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:54.150962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:54.212125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c97b9-28a2-432d-9585-0c656e05c8d0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:54.227233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:54.229361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:54.229787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:54.230235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] PATH_INFO: `/v3/projects/ab16dd3fec4e4154885efb716dfb2cd5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:54.232933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab16dd3fec4e4154885efb716dfb2cd5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:54.237796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:54.290124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99adec0-df18-4396-8f22-f1c841b333da None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:54.296101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:54.297766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:54.298929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:54.299312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:54.319163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:54.322252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:54.334768 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 973/1944] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:54.352496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab16dd3fec4e4154885efb716dfb2cd5', 'request_id': 'req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.362852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab16dd3fec4e4154885efb716dfb2cd5', 'request_id': 'req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.372333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab16dd3fec4e4154885efb716dfb2cd5', 'request_id': 'req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.387565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] The token cache is being invalidate because project ab16dd3fec4e4154885efb716dfb2cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:54.387895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.388669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.390285 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 972/1945] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:53 2022] DELETE /identity/v3/projects/ab16dd3fec4e4154885efb716dfb2cd5 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:54.395459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:54.409295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18367206-3fee-4ebf-897f-4d00817a051d service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:54.500728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:54.501304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:54.501590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:54.501868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] PATH_INFO: `/v3/users/cfc1ce1096f74f4398a93d105318d1be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:54.503132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc1ce1096f74f4398a93d105318d1be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:54.505166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:54.560950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:54.580432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc1ce1096f74f4398a93d105318d1be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.589343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc1ce1096f74f4398a93d105318d1be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.605292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc1ce1096f74f4398a93d105318d1be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.606980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc1ce1096f74f4398a93d105318d1be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.607864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc1ce1096f74f4398a93d105318d1be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:54.625237 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 974/1946] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:54 2022] DELETE /identity/v3/users/cfc1ce1096f74f4398a93d105318d1be => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:55.086958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:55.093381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:55.127328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2603b-a472-4d8b-bfbb-e9e03816c1cb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:55.205278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:55.223474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:55.225793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:55.226912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:55.227461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] PATH_INFO: `/v3/users/ed383d624e7849f780d5b8bfeec10607` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:55.233013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed383d624e7849f780d5b8bfeec10607)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:55.237420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:55.350222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:55.355327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:55.375029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62178a30-53c5-4397-a1e3-be0f7fc7f0a9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:55.380759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed383d624e7849f780d5b8bfeec10607'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.381856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:55.382402 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed383d624e7849f780d5b8bfeec10607'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.382766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:55.383056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:55.383386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] PATH_INFO: `/v3/projects/a634ca0915fc4f3b8be7f553467e9f36` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:55.384725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a634ca0915fc4f3b8be7f553467e9f36)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:55.387496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:55.407210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed383d624e7849f780d5b8bfeec10607'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.439745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed383d624e7849f780d5b8bfeec10607'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.443078 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5e7a1590-a9b8-4d66-b0cd-9e729ae98794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed383d624e7849f780d5b8bfeec10607'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.499857 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 973/1947] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:55 2022] DELETE /identity/v3/users/ed383d624e7849f780d5b8bfeec10607 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:55.501964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a634ca0915fc4f3b8be7f553467e9f36', 'request_id': 'req-9534eb96-3498-4b49-8a52-645e9663a8be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.504636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:55.505638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:55.506000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:55.511986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a634ca0915fc4f3b8be7f553467e9f36', 'request_id': 'req-9534eb96-3498-4b49-8a52-645e9663a8be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.520303 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a634ca0915fc4f3b8be7f553467e9f36', 'request_id': 'req-9534eb96-3498-4b49-8a52-645e9663a8be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.534912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:55.536642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:55.543955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] The token cache is being invalidate because project a634ca0915fc4f3b8be7f553467e9f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:55.544408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.545884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.552515 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 975/1948] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:55 2022] DELETE /identity/v3/projects/a634ca0915fc4f3b8be7f553467e9f36 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:55.555935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:55.559430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:55.635153 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 974/1949] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:55.649787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:55.669122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:55.706264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534eb96-3498-4b49-8a52-645e9663a8be admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:55.721372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:55.722122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:55.722499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:55.722982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] PATH_INFO: `/v3/users/f51fb49a88824a3ab6c4e6e858901bc9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:55.731533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51fb49a88824a3ab6c4e6e858901bc9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:55.748850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:55.804301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:55.813995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c17ea-f877-426b-96dd-03874bca1053 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:55.828347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:55.828915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:55.829276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:55.829941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:55.832939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51fb49a88824a3ab6c4e6e858901bc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.844660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:55.846308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:55.849065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51fb49a88824a3ab6c4e6e858901bc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.858545 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 975/1950] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:01:55.859333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:55.861269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:55.890597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b64ef-fb2d-4ec0-9b9d-37bd8a21544b service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:55.894626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51fb49a88824a3ab6c4e6e858901bc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.895433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51fb49a88824a3ab6c4e6e858901bc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.896069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8255130a-399e-45f5-b7b0-f4e2acdf4f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51fb49a88824a3ab6c4e6e858901bc9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:55.897769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:55.898263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:55.898731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:55.899011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] PATH_INFO: `/v3/projects/d4e029bb3d604ca5ae03e461c4e33e63` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:55.900363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e029bb3d604ca5ae03e461c4e33e63)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:55.902522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:55.924940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:55.929935 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 976/1951] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:55 2022] DELETE /identity/v3/users/f51fb49a88824a3ab6c4e6e858901bc9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:56.002653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e029bb3d604ca5ae03e461c4e33e63', 'request_id': 'req-a58f944c-0ba8-44e1-899c-ebf9a4985e98'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:56.017624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e029bb3d604ca5ae03e461c4e33e63', 'request_id': 'req-a58f944c-0ba8-44e1-899c-ebf9a4985e98'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:56.037914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e029bb3d604ca5ae03e461c4e33e63', 'request_id': 'req-a58f944c-0ba8-44e1-899c-ebf9a4985e98'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:56.058869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] The token cache is being invalidate because project d4e029bb3d604ca5ae03e461c4e33e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:56.059590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:56.060547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a58f944c-0ba8-44e1-899c-ebf9a4985e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:56.066251 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 976/1952] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:55 2022] DELETE /identity/v3/projects/d4e029bb3d604ca5ae03e461c4e33e63 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:56.070122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa7d2a-1b77-478b-9983-f1c7753bb283 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.070896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa7d2a-1b77-478b-9983-f1c7753bb283 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.071265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa7d2a-1b77-478b-9983-f1c7753bb283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.110541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-96fa7d2a-1b77-478b-9983-f1c7753bb283 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:56.111034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-96fa7d2a-1b77-478b-9983-f1c7753bb283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:56.228785 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 977/1953] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:56.249142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ff32f-fe55-4aba-a667-81b2f66e223a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.250029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ff32f-fe55-4aba-a667-81b2f66e223a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.250029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ff32f-fe55-4aba-a667-81b2f66e223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.291208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1c2ff32f-fe55-4aba-a667-81b2f66e223a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:56.293162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1c2ff32f-fe55-4aba-a667-81b2f66e223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:56.311721 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 977/1954] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:56.314417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa7d2a-1b77-478b-9983-f1c7753bb283 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:56.335534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa7d2a-1b77-478b-9983-f1c7753bb283 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:56.347792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:56.348363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.349004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.349391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.350562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:56.352029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:56.355340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:56.379586 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 978/1955] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:01:56.385717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ff32f-fe55-4aba-a667-81b2f66e223a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:56.410440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ff32f-fe55-4aba-a667-81b2f66e223a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:56.419363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:56.420418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.420897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.421239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.421989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:56.424267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:56.495780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:56.504664 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 978/1956] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:56 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:56.532845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44de52-1368-4546-a57c-8f283af08bee admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:56.618177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:56.620475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:56.699760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:56.700440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.700729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.701001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.701652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:56.705216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:56.764085 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 979/1957] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:56 2022] POST /identity/v3/users => generated 282 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:56.769486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:56.787334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8dbc4-d8ac-4ccf-b35b-26f745f25d91 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:56.805893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:56.806568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.806852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.807134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] PATH_INFO: `/v3/projects/db2371d19d88445f9bb447e3a6e49eae` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.812038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2371d19d88445f9bb447e3a6e49eae)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:56.814302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:56.815532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada435a8-de59-4c00-8927-6e48befc41eb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:56.823478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:56.824347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.825046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.825564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.826497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:56.828494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:56.844022 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 980/1958] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:56.849987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:56.870564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765b95b-80f3-44ee-8132-808554593a8e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:56.890985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:56.892217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:56.892619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:56.893371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:56.894401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:56.901871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:56.952028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2371d19d88445f9bb447e3a6e49eae', 'request_id': 'req-a27edb35-535e-4b5c-8742-976f4bad1a1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:56.967519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2371d19d88445f9bb447e3a6e49eae', 'request_id': 'req-a27edb35-535e-4b5c-8742-976f4bad1a1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:56.973012 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 981/1959] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:56.977041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:56.996481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2371d19d88445f9bb447e3a6e49eae', 'request_id': 'req-a27edb35-535e-4b5c-8742-976f4bad1a1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.000143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:57.000451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:57.011422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] The token cache is being invalidate because project db2371d19d88445f9bb447e3a6e49eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:57.012066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.017471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.029440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:57.030562 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 979/1960] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:56 2022] DELETE /identity/v3/projects/db2371d19d88445f9bb447e3a6e49eae => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:01:57.038873 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:57.042247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a761d4-d681-4cff-a26b-27aba6c1360d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:57.046745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:57.167402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:57.168191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:57.168698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:57.169190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] PATH_INFO: `/v3/projects/395beb87b81944e395ed04763fc6a0a7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:57.171734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=395beb87b81944e395ed04763fc6a0a7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:57.175375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:57.218560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edb35-535e-4b5c-8742-976f4bad1a1c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:57.228951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:57.229964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:57.230313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:57.230684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] PATH_INFO: `/v3/projects/6f12f7dd38294124ba36bcdbeb0a033b/users/c4247d0d7bdd4c13a40237a1f183d3b7/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:57.266231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f12f7dd38294124ba36bcdbeb0a033b, user_id=c4247d0d7bdd4c13a40237a1f183d3b7, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:57.271309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:57.274307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395beb87b81944e395ed04763fc6a0a7', 'request_id': 'req-72613b36-a66a-415b-86dc-e8c3ade2e4bc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.292957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395beb87b81944e395ed04763fc6a0a7', 'request_id': 'req-72613b36-a66a-415b-86dc-e8c3ade2e4bc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.295565 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 980/1961] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:57 2022] PUT /identity/v3/projects/6f12f7dd38294124ba36bcdbeb0a033b/users/c4247d0d7bdd4c13a40237a1f183d3b7/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:57.297356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:57.314569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:57.317589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395beb87b81944e395ed04763fc6a0a7', 'request_id': 'req-72613b36-a66a-415b-86dc-e8c3ade2e4bc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.361951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] The token cache is being invalidate because project 395beb87b81944e395ed04763fc6a0a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:57.362633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.364316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-72613b36-a66a-415b-86dc-e8c3ade2e4bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.367405 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 982/1962] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:56 2022] DELETE /identity/v3/projects/395beb87b81944e395ed04763fc6a0a7 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:57.369562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927140-08e4-475b-bc5f-29d7f5ab0d8a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:57.370038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927140-08e4-475b-bc5f-29d7f5ab0d8a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:57.370529 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927140-08e4-475b-bc5f-29d7f5ab0d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:57.422382 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba3e1-4a7b-4d69-9a4e-2788efec8401 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:57.430066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:57.431280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c9927140-08e4-475b-bc5f-29d7f5ab0d8a None None] MFA Rules not processed for user `c4247d0d7bdd4c13a40237a1f183d3b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:57.431458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:57.431993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:57.432622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] PATH_INFO: `/v3/users/10e15a75ebc6465abf41ba3e6a1873e9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:57.433612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9927140-08e4-475b-bc5f-29d7f5ab0d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:57.435488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e15a75ebc6465abf41ba3e6a1873e9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:57.440477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:57.528326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e15a75ebc6465abf41ba3e6a1873e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.529453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e15a75ebc6465abf41ba3e6a1873e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.541991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e15a75ebc6465abf41ba3e6a1873e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.560083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e15a75ebc6465abf41ba3e6a1873e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.561985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e15a75ebc6465abf41ba3e6a1873e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:57.578339 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 983/1963] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:57.581089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511ec0d-45cf-4908-baa8-7654d51bae9f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:57.581533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511ec0d-45cf-4908-baa8-7654d51bae9f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:57.582792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511ec0d-45cf-4908-baa8-7654d51bae9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:57.588344 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 981/1964] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:01:57 2022] DELETE /identity/v3/users/10e15a75ebc6465abf41ba3e6a1873e9 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:57.594272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:57.596590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:57.604907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:57.630739 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a511ec0d-45cf-4908-baa8-7654d51bae9f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:57.633185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a511ec0d-45cf-4908-baa8-7654d51bae9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:57.688354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:57.840831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555bcd7-6f89-4485-831f-46b820b723bf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:57.845295 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 984/1965] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:57.857174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:57.859559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:57.865045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:57.865939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:57.867208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:57.871956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383ca35-54e1-45a9-8c27-730206fa65fd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:57.872662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383ca35-54e1-45a9-8c27-730206fa65fd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:57.873098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:57.873262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383ca35-54e1-45a9-8c27-730206fa65fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:57.924513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e383ca35-54e1-45a9-8c27-730206fa65fd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:57.926879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e383ca35-54e1-45a9-8c27-730206fa65fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:57.945153 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 985/1966] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:01:57.947645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383ca35-54e1-45a9-8c27-730206fa65fd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:57.948598 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 982/1967] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:57 2022] POST /identity/v3/projects => generated 365 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:57.950258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:57.961686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383ca35-54e1-45a9-8c27-730206fa65fd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.089243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.092516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.093162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.095136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.096239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.103616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51130d5-2103-47b1-9b08-5627c8a6fceb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.105401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.113719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.115311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.115311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.116063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.117525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:58.118133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.122052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.179140 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 983/1968] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:58.181465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.185511 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 986/1969] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:57 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:01:58.199584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.206886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c0434-8672-47c5-9355-595772af81ae admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.235105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e15fee-dc3a-4a38-84bb-00b338abf219 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.239543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.247426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.248509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.249533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.249993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.250175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.250452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.250715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.251410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.256176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.259236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.268751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.287512 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 987/1970] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:58.294654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.323342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c8d64-405e-43f2-999b-ef1498b60d45 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.334484 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.350189 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.351051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.351637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] PATH_INFO: `/v3/projects/9984f630cc6e48bbb88d3de67f5fa8f6/users/d138c2026ee748448f2f860fdfeec889/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.359906 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 984/1971] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:58 2022] POST /identity/v3/projects => generated 389 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:58.367175 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.414629 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9984f630cc6e48bbb88d3de67f5fa8f6, user_id=d138c2026ee748448f2f860fdfeec889, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.423660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.479451 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 988/1972] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:58 2022] PUT /identity/v3/projects/9984f630cc6e48bbb88d3de67f5fa8f6/users/d138c2026ee748448f2f860fdfeec889/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:58.485948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.505298 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27338989-0011-4589-80e6-9d0db5c4914c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.521613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:58.636715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.639074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.639436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.639783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.640821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.643690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.681467 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 985/1973] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:01:58 2022] POST /identity/v3/users => generated 295 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:58.683999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.686914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:58.717187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f664c6-99d6-4eed-bdb9-6420fcddca78 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.718108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6e1aa-5e1a-4363-81e7-8d8bc66210e4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.727130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.727616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.728220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.728777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.728951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.729267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.729469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.729717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] PATH_INFO: `/v3/projects/f155ce0c2a784e8c888a595ad7456185` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.730518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.732027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f155ce0c2a784e8c888a595ad7456185)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.734795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.736107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.755690 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 989/1974] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:58.758208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.775893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6adaf8-d82b-46ac-bf9e-e428df1c2778 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.784001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.785011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.785320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:58.785598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:58.786415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:58.789079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:58.792780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:58.827077 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 990/1975] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:58.829005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.847975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcecb4-3994-4bb1-b2cf-61b732c312b8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:58.848994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f155ce0c2a784e8c888a595ad7456185', 'request_id': 'req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:58.868502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f155ce0c2a784e8c888a595ad7456185', 'request_id': 'req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:58.878368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f155ce0c2a784e8c888a595ad7456185', 'request_id': 'req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:58.906489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] The token cache is being invalidate because project f155ce0c2a784e8c888a595ad7456185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:01:58.906947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:58.908148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:01:58.910275 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 986/1976] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:01:58 2022] DELETE /identity/v3/projects/f155ce0c2a784e8c888a595ad7456185 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:58.911321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:58.913849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:58.997799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd59759-bb04-459e-b2f6-ca961a4a4042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:58.999385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd59759-bb04-459e-b2f6-ca961a4a4042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:58.999900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd59759-bb04-459e-b2f6-ca961a4a4042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.000465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd59759-bb04-459e-b2f6-ca961a4a4042 admin admin] PATH_INFO: `/v3/projects/9984f630cc6e48bbb88d3de67f5fa8f6/users/d138c2026ee748448f2f860fdfeec889/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.005740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd59759-bb04-459e-b2f6-ca961a4a4042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9984f630cc6e48bbb88d3de67f5fa8f6, user_id=d138c2026ee748448f2f860fdfeec889, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:59.013898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd59759-bb04-459e-b2f6-ca961a4a4042 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:59.018149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a8fde-837f-4f49-9e4d-9e80a0fd3378 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:59.026240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b929d7a-0166-45fe-80ac-c17fdebb01b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:59.026894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929d7a-0166-45fe-80ac-c17fdebb01b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.028155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929d7a-0166-45fe-80ac-c17fdebb01b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.028155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929d7a-0166-45fe-80ac-c17fdebb01b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.029062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b929d7a-0166-45fe-80ac-c17fdebb01b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:59.031740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b929d7a-0166-45fe-80ac-c17fdebb01b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:59.042067 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 991/1977] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:58 2022] PUT /identity/v3/projects/9984f630cc6e48bbb88d3de67f5fa8f6/users/d138c2026ee748448f2f860fdfeec889/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:59.058815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd4f60-0bc6-48e2-adf5-e94b6dc755a5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.059321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd4f60-0bc6-48e2-adf5-e94b6dc755a5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.059821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd4f60-0bc6-48e2-adf5-e94b6dc755a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.071420 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 987/1978] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:01:59.074781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd9fca-982a-4caa-a825-149efc987c3a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.076728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd9fca-982a-4caa-a825-149efc987c3a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.078972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd9fca-982a-4caa-a825-149efc987c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.123766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fffd4f60-0bc6-48e2-adf5-e94b6dc755a5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:59.126339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fffd4f60-0bc6-48e2-adf5-e94b6dc755a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:59.185416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a6bd9fca-982a-4caa-a825-149efc987c3a None None] MFA Rules not processed for user `d138c2026ee748448f2f860fdfeec889`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:59.187461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a6bd9fca-982a-4caa-a825-149efc987c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:59.197798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:59.318425 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 992/1979] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:59.320211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd4f60-0bc6-48e2-adf5-e94b6dc755a5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:59.361906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd4f60-0bc6-48e2-adf5-e94b6dc755a5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:59.363612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba86c64-5455-48a9-a325-39a0138474b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:59.368584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba86c64-5455-48a9-a325-39a0138474b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.368584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba86c64-5455-48a9-a325-39a0138474b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.368584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba86c64-5455-48a9-a325-39a0138474b1 admin admin] PATH_INFO: `/v3/projects/4b6dcb395879477ebf5dca122b7e5a56/users/01e0fc641bf546b495923fb52b3e1258/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.414935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba86c64-5455-48a9-a325-39a0138474b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6dcb395879477ebf5dca122b7e5a56, user_id=01e0fc641bf546b495923fb52b3e1258, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:59.418817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba86c64-5455-48a9-a325-39a0138474b1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:59.474471 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 993/1980] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:01:59 2022] PUT /identity/v3/projects/4b6dcb395879477ebf5dca122b7e5a56/users/01e0fc641bf546b495923fb52b3e1258/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:01:59.477882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.478284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.478892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.493555 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 988/1981] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:59.503689 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61442a8-9d58-497b-816c-2e8dcecaa71c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.503689 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61442a8-9d58-497b-816c-2e8dcecaa71c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.503689 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61442a8-9d58-497b-816c-2e8dcecaa71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.512428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:59.516795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:59.567539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b61442a8-9d58-497b-816c-2e8dcecaa71c None None] MFA Rules not processed for user `01e0fc641bf546b495923fb52b3e1258`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:01:59.569008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b61442a8-9d58-497b-816c-2e8dcecaa71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:59.628509 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 994/1982] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:59.634360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:59.637191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:01:59.655700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:59.700504 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 989/1983] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:01:59.700504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61442a8-9d58-497b-816c-2e8dcecaa71c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:59.726851 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61442a8-9d58-497b-816c-2e8dcecaa71c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:59.760330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:59.761585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.762052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.762508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.763695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:01:59.766985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:59.769881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:59.788019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87ebab-0194-4a1e-b1dd-a497ff3a3188 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:59.810339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:59.811927 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 990/1984] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:01:59.813418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:01:59.815307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.816463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.816962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.838393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941cf67-c33d-4f9d-b697-8734ce3bdb35 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:01:59.848614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:01:59.849850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:01:59.850296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:01:59.853230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:01:59.854570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:01:59.858766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:01:59.965909 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 991/1985] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:59 2022] POST /identity/v3/projects => generated 389 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:01:59.968051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.030581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.032351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.059389 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 995/1986] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:01:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:00.060745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.084665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e675a5b-4889-4cce-a75e-7b1f93ce929c service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.193545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:00.204361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae911e-1de3-4415-88df-bfa541f37982 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.221723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.225148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.228449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.229003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.233595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.240254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.278293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.282643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.284643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.287260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.287463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.291072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.305676 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 992/1987] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:01:59 2022] POST /identity/v3/projects => generated 387 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:00.317379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.344609 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 996/1988] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:00 2022] POST /identity/v3/users => generated 301 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:00.344609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.426704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d298821-d859-4d36-8158-eef6cb194e3d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.433741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.434507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.434789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.435110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.435846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.437898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.443872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:00.457337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e2869-7ef1-44a2-a671-e28cd5bc410d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.475770 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 993/1989] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:00 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:00.477981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.497350 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad8312-b633-4883-a9d7-aa808be13aa8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.525560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.527155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.528949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.529238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.529994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.535270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.555049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.555713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.555987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.556271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.559336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:00.572191 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 994/1990] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:00.574277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.597283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73545ca8-a229-4bdb-abf9-59648111d8e4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.609393 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.610660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.611276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.611912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.613417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.617548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.646729 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 995/1991] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:00.649215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.666727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.668867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.684745 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 997/1992] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:00.687179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.688492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70bb1f-6e16-42f5-ae9a-2b19a30d33b4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.694974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.695878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.696300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.696832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] PATH_INFO: `/v3/projects/e0bd2b74ea194ef2a27b0921d619806d/users/919bf74e731f42a99ae6d8655efb62da/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.704238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df21dd-318e-4bfe-a158-c697e77af5e6 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.712033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.712961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.713428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.713876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.715136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.718016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.732893 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 998/1993] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:00.738654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.739871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bd2b74ea194ef2a27b0921d619806d, user_id=919bf74e731f42a99ae6d8655efb62da, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.745507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.763246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc65430-769a-4bf0-9767-fc31e4e8eb6f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:00.785676 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 996/1994] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:00 2022] PUT /identity/v3/projects/e0bd2b74ea194ef2a27b0921d619806d/users/919bf74e731f42a99ae6d8655efb62da/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:00.787151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9a9b0-ee1c-4b3f-a9a8-7f487b5645e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:00.792662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:00.793945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9a9b0-ee1c-4b3f-a9a8-7f487b5645e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.793945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9a9b0-ee1c-4b3f-a9a8-7f487b5645e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.809122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9a9b0-ee1c-4b3f-a9a8-7f487b5645e6 admin admin] PATH_INFO: `/v3/projects/ebefd5e22ebc4fd28a6785beb838f3be/users/d2066168be024770ba0593713c9741f0/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.875352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9a9b0-ee1c-4b3f-a9a8-7f487b5645e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebefd5e22ebc4fd28a6785beb838f3be, user_id=d2066168be024770ba0593713c9741f0, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:00.887392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9a9b0-ee1c-4b3f-a9a8-7f487b5645e6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:00.928619 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 999/1995] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:00 2022] PUT /identity/v3/projects/ebefd5e22ebc4fd28a6785beb838f3be/users/d2066168be024770ba0593713c9741f0/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:00.970516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d27a-2251-4efd-ae78-7e4490514e6f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:00.973614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d27a-2251-4efd-ae78-7e4490514e6f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:00.974322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0d27a-2251-4efd-ae78-7e4490514e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:00.976245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bcddf-608e-4b83-af15-7f9578ceeca5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:01.111316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f2c0d27a-2251-4efd-ae78-7e4490514e6f None None] MFA Rules not processed for user `d2066168be024770ba0593713c9741f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:01.114696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f2c0d27a-2251-4efd-ae78-7e4490514e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:01.126922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:01.204456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:01.206086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.206497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.206917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.208707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:01.211241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:01.225722 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 997/1996] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:01.230583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:01.245122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2db57-2dc7-46f1-97d6-236879d23251 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:01.253745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da442f71-781d-47bd-bee4-8717256cc4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:01.256596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da442f71-781d-47bd-bee4-8717256cc4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.257117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da442f71-781d-47bd-bee4-8717256cc4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.257504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da442f71-781d-47bd-bee4-8717256cc4fc admin admin] PATH_INFO: `/v3/projects/e0bd2b74ea194ef2a27b0921d619806d/users/919bf74e731f42a99ae6d8655efb62da/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.261819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da442f71-781d-47bd-bee4-8717256cc4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bd2b74ea194ef2a27b0921d619806d, user_id=919bf74e731f42a99ae6d8655efb62da, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:01.268504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da442f71-781d-47bd-bee4-8717256cc4fc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:01.296854 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 998/1997] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:01 2022] PUT /identity/v3/projects/e0bd2b74ea194ef2a27b0921d619806d/users/919bf74e731f42a99ae6d8655efb62da/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:01.309940 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1000/1998] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:01.313328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90505d7-2fc2-4975-a091-5dde100be0ec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.313782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90505d7-2fc2-4975-a091-5dde100be0ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.314004 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90505d7-2fc2-4975-a091-5dde100be0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.318626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0d27a-2251-4efd-ae78-7e4490514e6f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:01.355594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0d27a-2251-4efd-ae78-7e4490514e6f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:01.389430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:01.392638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b90505d7-2fc2-4975-a091-5dde100be0ec None None] MFA Rules not processed for user `919bf74e731f42a99ae6d8655efb62da`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:01.392638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b90505d7-2fc2-4975-a091-5dde100be0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:01.506400 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 999/1999] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:01.515660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:01.517066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.517495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.517688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.518307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:01.518473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90505d7-2fc2-4975-a091-5dde100be0ec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:01.520971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:01.575571 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1001/2000] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:01 2022] POST /identity/v3/projects => generated 387 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:01.578104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:01.622616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90505d7-2fc2-4975-a091-5dde100be0ec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:01.707168 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054d0b6-5735-4c94-99d0-8a2a8f18568d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:01.713446 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:01.714124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.714454 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.714689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.715089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:01.715338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:01.716364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.717162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.717651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:01.717756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.735996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:01.746758 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1002/2001] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:01 2022] POST /identity/v3/users => generated 300 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:01.751693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:01.766232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe791d64-a147-4e42-ac72-e0b8df9e9d88 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:01.774700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:01.775391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.776718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.776718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.776718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:01.778305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:01.791705 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1003/2002] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:01.798553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:01.820620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303febf-04ff-4ffb-b176-8bcb69691627 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:01.825570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:01.827042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:01.829717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:01.830844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:01.831336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:01.832022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] PATH_INFO: `/v3/projects/fc4328beccbe41bb92975515308a5e4d/users/897ef27cd7344f9d8fcb06b6127f8fb2/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:01.839005 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1000/2003] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:01 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:01.871681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4328beccbe41bb92975515308a5e4d, user_id=897ef27cd7344f9d8fcb06b6127f8fb2, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:01.875377 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:01.903030 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1004/2004] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:01 2022] PUT /identity/v3/projects/fc4328beccbe41bb92975515308a5e4d/users/897ef27cd7344f9d8fcb06b6127f8fb2/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:01.906584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:01.935343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9dce64-210e-4273-bcc3-ff9287869f91 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:02.018166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:02.050153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:02.051233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:02.053384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:02.053384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:02.053660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:02.057226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:02.079789 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1001/2005] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:02:02.084734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:02.113259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53932dac-b1b4-4571-813c-fcff1fbc690d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:02.123889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079198a-2ceb-4540-9fa1-aeab02bbd3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:02.124821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079198a-2ceb-4540-9fa1-aeab02bbd3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:02.125433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079198a-2ceb-4540-9fa1-aeab02bbd3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:02.126063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079198a-2ceb-4540-9fa1-aeab02bbd3a5 admin admin] PATH_INFO: `/v3/projects/fc4328beccbe41bb92975515308a5e4d/users/897ef27cd7344f9d8fcb06b6127f8fb2/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:02.130035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079198a-2ceb-4540-9fa1-aeab02bbd3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4328beccbe41bb92975515308a5e4d, user_id=897ef27cd7344f9d8fcb06b6127f8fb2, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:02.136323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079198a-2ceb-4540-9fa1-aeab02bbd3a5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:02.180325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:02.181769 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1002/2006] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:02 2022] PUT /identity/v3/projects/fc4328beccbe41bb92975515308a5e4d/users/897ef27cd7344f9d8fcb06b6127f8fb2/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:02.211037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:02.214603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:02.215559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:02.229654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed99fa-70d2-4575-a59d-97f6ae29fb25 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:02.291131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] MFA Rules not processed for user `897ef27cd7344f9d8fcb06b6127f8fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:02.293155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:02.415017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:02.416141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:02.416734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:02.417343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] PATH_INFO: `/v3/users/01e0fc641bf546b495923fb52b3e1258` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:02.419171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e0fc641bf546b495923fb52b3e1258)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:02.424140 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:02.519982 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1003/2007] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:02.531059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:02.542125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:02.556293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e0fc641bf546b495923fb52b3e1258'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:02.578289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e0fc641bf546b495923fb52b3e1258'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:02.612440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:02.641653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e0fc641bf546b495923fb52b3e1258'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:02.642899 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e0fc641bf546b495923fb52b3e1258'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:02.643896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d25713c7-6714-4e30-931d-b0e21130dd00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e0fc641bf546b495923fb52b3e1258'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:02.677862 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1005/2008] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:02 2022] DELETE /identity/v3/users/01e0fc641bf546b495923fb52b3e1258 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:02.686017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:02.686678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:02.687137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:02.712707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e3191-1e63-4ffb-bdc6-e939118f5460 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:02.728272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:02.730723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:02.794578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:02.795353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:02.795889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:02.796188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:02.801242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:02.882894 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1006/2009] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:02.898528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:02.901147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:02.950823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:03.044258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edd7ea-1001-4460-abb3-c1b7f0bfd0f4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:03.051936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ef76a-ba41-437b-90fb-ce804a19e5ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:03.053059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef76a-ba41-437b-90fb-ce804a19e5ff service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:03.053578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef76a-ba41-437b-90fb-ce804a19e5ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:03.054587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ef76a-ba41-437b-90fb-ce804a19e5ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:03.057113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:03.059189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:03.074111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ef76a-ba41-437b-90fb-ce804a19e5ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:03.077052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ef76a-ba41-437b-90fb-ce804a19e5ff service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:03.087611 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1004/2010] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:03.114877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:03.116493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:03.121167 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1007/2011] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:03.144373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11851291-b0db-4bb6-8d8c-dab840aabed1 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:03.155000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:03.156000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:03.157944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:03.157944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] PATH_INFO: `/v3/users/d2066168be024770ba0593713c9741f0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:03.159643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2066168be024770ba0593713c9741f0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:03.164698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:03.301864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2066168be024770ba0593713c9741f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.304087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2066168be024770ba0593713c9741f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.320929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2066168be024770ba0593713c9741f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.351859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2066168be024770ba0593713c9741f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.356747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2066168be024770ba0593713c9741f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.363148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:03.405824 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1005/2012] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:03 2022] DELETE /identity/v3/users/d2066168be024770ba0593713c9741f0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:03.411742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:03.412103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:03.412295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:03.452107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:03.453949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:03.617709 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1008/2013] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:03.626636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:03.631306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:03.675608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:03.677301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:03.696032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270a6ed-2d77-4921-8054-7350c5057463 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:03.708521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:03.709607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:03.709894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:03.710944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] PATH_INFO: `/v3/projects/4b6dcb395879477ebf5dca122b7e5a56` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:03.712881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6dcb395879477ebf5dca122b7e5a56)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:03.716121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:03.780825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79464ead-387c-451b-920c-644bbceaf6ca admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:03.803140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:03.806378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:03.807171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:03.807915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:03.824732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:03.827023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6dcb395879477ebf5dca122b7e5a56', 'request_id': 'req-27be37eb-e661-4b48-b40e-67c0e013f972'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.827312 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:03.840373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6dcb395879477ebf5dca122b7e5a56', 'request_id': 'req-27be37eb-e661-4b48-b40e-67c0e013f972'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.843420 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1006/2014] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:03.852334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6dcb395879477ebf5dca122b7e5a56', 'request_id': 'req-27be37eb-e661-4b48-b40e-67c0e013f972'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.880042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] The token cache is being invalidate because project 4b6dcb395879477ebf5dca122b7e5a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:03.880584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.881860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:03.884976 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1009/2015] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:03 2022] DELETE /identity/v3/projects/4b6dcb395879477ebf5dca122b7e5a56 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:03.891680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:03.916157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab3486-c3c3-401b-9c74-493cf818461f service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:04.042202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:04.043313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:04.043929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:04.044665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] PATH_INFO: `/v3/users/919bf74e731f42a99ae6d8655efb62da` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:04.047305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=919bf74e731f42a99ae6d8655efb62da)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:04.054035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:04.238424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919bf74e731f42a99ae6d8655efb62da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.240755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919bf74e731f42a99ae6d8655efb62da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.258225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919bf74e731f42a99ae6d8655efb62da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.288665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919bf74e731f42a99ae6d8655efb62da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.289675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919bf74e731f42a99ae6d8655efb62da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.326778 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1007/2016] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:03 2022] DELETE /identity/v3/users/919bf74e731f42a99ae6d8655efb62da => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:04.520547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:04.523477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:04.547921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be37eb-e661-4b48-b40e-67c0e013f972 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:04.656057 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:04.702710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:04.703529 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:04.703967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:04.704619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] PATH_INFO: `/v3/projects/ebefd5e22ebc4fd28a6785beb838f3be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:04.705775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebefd5e22ebc4fd28a6785beb838f3be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:04.708536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:04.785248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebefd5e22ebc4fd28a6785beb838f3be', 'request_id': 'req-f866a137-7f62-478e-9c91-05a37dbbed56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.807311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:04.809147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:04.813453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebefd5e22ebc4fd28a6785beb838f3be', 'request_id': 'req-f866a137-7f62-478e-9c91-05a37dbbed56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.825875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f9c59-0c47-4427-8fcb-558a567c4d96 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:04.834579 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebefd5e22ebc4fd28a6785beb838f3be', 'request_id': 'req-f866a137-7f62-478e-9c91-05a37dbbed56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.840113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:04.875875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] The token cache is being invalidate because project ebefd5e22ebc4fd28a6785beb838f3be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:04.876176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.877075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:04.881340 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1010/2017] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:04 2022] DELETE /identity/v3/projects/ebefd5e22ebc4fd28a6785beb838f3be => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:04.891729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:04.899207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:04.937925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:04.940090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:04.940701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:04.941217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] PATH_INFO: `/v3/users/c4247d0d7bdd4c13a40237a1f183d3b7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:04.943951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4247d0d7bdd4c13a40237a1f183d3b7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:04.947696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:05.041294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4247d0d7bdd4c13a40237a1f183d3b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.042149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4247d0d7bdd4c13a40237a1f183d3b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.081856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4247d0d7bdd4c13a40237a1f183d3b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.086528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866a137-7f62-478e-9c91-05a37dbbed56 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:05.110819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:05.112496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:05.112972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:05.113329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] PATH_INFO: `/v3/users/897ef27cd7344f9d8fcb06b6127f8fb2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:05.115922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] RBAC: Authorizing `identity:delete_user(user_id=897ef27cd7344f9d8fcb06b6127f8fb2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:05.123397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:05.130667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4247d0d7bdd4c13a40237a1f183d3b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.131827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-79aec883-661e-4285-b988-a03038efe673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4247d0d7bdd4c13a40237a1f183d3b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.208238 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1008/2018] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:04 2022] DELETE /identity/v3/users/c4247d0d7bdd4c13a40237a1f183d3b7 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:05.215978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:05.216316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:05.216652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:05.261268 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:05.261758 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '897ef27cd7344f9d8fcb06b6127f8fb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.264787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:05.283776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '897ef27cd7344f9d8fcb06b6127f8fb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.321146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '897ef27cd7344f9d8fcb06b6127f8fb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.324318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '897ef27cd7344f9d8fcb06b6127f8fb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.330366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '897ef27cd7344f9d8fcb06b6127f8fb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:05.339115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:05.358517 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1011/2019] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:04 2022] DELETE /identity/v3/users/897ef27cd7344f9d8fcb06b6127f8fb2 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:05.462073 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1009/2020] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:05.483394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:05.486134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:05.643010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f2faf-a235-4e58-9bbf-f4b40fd1412e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:05.650625 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:05.651830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:05.652800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:05.653104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:05.828347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:05.831122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:05.852856 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1012/2021] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:06.078719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:06.082770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:06.111590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fb95-00b2-48b4-9b5a-afc6567b1f37 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:06.142995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:06.144150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:06.145178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:06.146383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] PATH_INFO: `/v3/projects/e0bd2b74ea194ef2a27b0921d619806d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:06.149889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0bd2b74ea194ef2a27b0921d619806d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:06.155621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:06.261196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:06.263368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:06.328948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f2a12-aa01-4c87-9811-0c4e401dcd79 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:06.346806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:06.349727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:06.351496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:06.352218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:06.352643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] PATH_INFO: `/v3/projects/fc4328beccbe41bb92975515308a5e4d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:06.355460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4328beccbe41bb92975515308a5e4d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:06.358428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:06.383210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0bd2b74ea194ef2a27b0921d619806d', 'request_id': 'req-70a4f6f9-4778-4172-9905-33aec2c5b62a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.416439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0bd2b74ea194ef2a27b0921d619806d', 'request_id': 'req-70a4f6f9-4778-4172-9905-33aec2c5b62a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.438366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0bd2b74ea194ef2a27b0921d619806d', 'request_id': 'req-70a4f6f9-4778-4172-9905-33aec2c5b62a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.472408 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] The token cache is being invalidate because project e0bd2b74ea194ef2a27b0921d619806d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:06.493500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4328beccbe41bb92975515308a5e4d', 'request_id': 'req-64029359-23b6-45bb-a544-a15679b64df4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.505290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.509976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.515018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4328beccbe41bb92975515308a5e4d', 'request_id': 'req-64029359-23b6-45bb-a544-a15679b64df4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.518807 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1010/2022] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:06 2022] DELETE /identity/v3/projects/e0bd2b74ea194ef2a27b0921d619806d => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:06.521232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:06.540520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:06.544096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4328beccbe41bb92975515308a5e4d', 'request_id': 'req-64029359-23b6-45bb-a544-a15679b64df4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.593944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] The token cache is being invalidate because project fc4328beccbe41bb92975515308a5e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:06.594421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.595618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64029359-23b6-45bb-a544-a15679b64df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:06.608870 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1013/2023] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:06 2022] DELETE /identity/v3/projects/fc4328beccbe41bb92975515308a5e4d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:06.608870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6254-6cb8-4e11-a3af-e652b18eb70a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:06.608870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6254-6cb8-4e11-a3af-e652b18eb70a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:06.608870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c6254-6cb8-4e11-a3af-e652b18eb70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:06.657289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-312c6254-6cb8-4e11-a3af-e652b18eb70a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:06.659586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-312c6254-6cb8-4e11-a3af-e652b18eb70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:06.720828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:06.842963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f6f9-4778-4172-9905-33aec2c5b62a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:06.858683 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1014/2024] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:06.859244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:06.859244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:06.859244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:06.859244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] PATH_INFO: `/v3/projects/6f12f7dd38294124ba36bcdbeb0a033b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:06.859591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbc316-d9d6-482b-a51d-ea419b8ec057 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:06.859591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbc316-d9d6-482b-a51d-ea419b8ec057 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:06.859591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbc316-d9d6-482b-a51d-ea419b8ec057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:06.861123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f12f7dd38294124ba36bcdbeb0a033b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:06.864707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:06.894419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bbfbc316-d9d6-482b-a51d-ea419b8ec057 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:06.896550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bbfbc316-d9d6-482b-a51d-ea419b8ec057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:06.908923 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1015/2025] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:06.911693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:06.912036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:06.912546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:06.957331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:06.961960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:07.023138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f12f7dd38294124ba36bcdbeb0a033b', 'request_id': 'req-7628de9c-f3ef-43d9-b934-1aa3cde89568'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.050010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f12f7dd38294124ba36bcdbeb0a033b', 'request_id': 'req-7628de9c-f3ef-43d9-b934-1aa3cde89568'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.081094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f12f7dd38294124ba36bcdbeb0a033b', 'request_id': 'req-7628de9c-f3ef-43d9-b934-1aa3cde89568'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.108118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] The token cache is being invalidate because project 6f12f7dd38294124ba36bcdbeb0a033b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:07.111597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.112990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7628de9c-f3ef-43d9-b934-1aa3cde89568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.119990 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1011/2026] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:06 2022] DELETE /identity/v3/projects/6f12f7dd38294124ba36bcdbeb0a033b => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:07.122325 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f0cd-2368-4ef9-b78d-57e8329475cf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.122663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f0cd-2368-4ef9-b78d-57e8329475cf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.122939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f0cd-2368-4ef9-b78d-57e8329475cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.169732 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1016/2027] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:07.171217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.175631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:07.202774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b158f0cd-2368-4ef9-b78d-57e8329475cf None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:07.206156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b158f0cd-2368-4ef9-b78d-57e8329475cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:07.216278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc68476-8676-41d3-bb50-481bbd4ac725 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.225490 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1012/2028] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:07.227158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158f0cd-2368-4ef9-b78d-57e8329475cf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.235567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.236561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.236846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.237120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] PATH_INFO: `/v3/users/d138c2026ee748448f2f860fdfeec889` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.239707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d138c2026ee748448f2f860fdfeec889)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.243775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:07.249770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158f0cd-2368-4ef9-b78d-57e8329475cf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.262743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.263976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.264515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.264995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.266511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:07.267257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.270716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:07.306106 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1013/2029] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:07.310037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.329249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540ad06-789c-4fce-b7bb-9e214d4321f3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.334413 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.335724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.336023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.336296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.337115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:07.337476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.339333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd138c2026ee748448f2f860fdfeec889'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.339550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:07.355004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd138c2026ee748448f2f860fdfeec889'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.356901 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1014/2030] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:07.361674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.374240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b3adf-2aa8-4f47-9bda-dddd89240ac8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.374532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd138c2026ee748448f2f860fdfeec889'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.375243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd138c2026ee748448f2f860fdfeec889'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.375982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd138c2026ee748448f2f860fdfeec889'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:07.380739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.381565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.381989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.382398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.383217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.385703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:07.394889 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1017/2031] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:07 2022] DELETE /identity/v3/users/d138c2026ee748448f2f860fdfeec889 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:07.396374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.399397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:07.402884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:07.438847 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1015/2032] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:07 2022] POST /identity/v3/projects => generated 409 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:07.446758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.449725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:07.481649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:07.544240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b695794-d16d-4d41-aaba-2ace741f207a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.597902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06366704-8617-4c5a-aa73-a81c96200496 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.607497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.609161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.609484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.609974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.611295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.614920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:07.675072 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1016/2033] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:07 2022] POST /identity/v3/users => generated 305 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:07.676916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.685780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.687151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.688763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.688763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.690034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.695065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:07.725403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cee9f-c61d-46ad-a1d6-d99862bd9a48 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.738040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.739287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.740984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.743772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.746551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.748523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:07.766690 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1018/2034] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:07 2022] POST /identity/v3/projects => generated 373 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:07.766991 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1017/2035] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:07.780832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.791123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:07.797233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec0aee-ad6a-4ee8-bc66-38d5200de5df admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.955015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29149310-d39a-4a1b-8695-d73c58336974 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:07.965319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:07.966525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:07.967004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:07.967481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:07.969819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:07.975398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:08.027895 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1019/2036] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:07 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:08.029887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:08.039494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:08.041151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.041877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.042692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.044690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:08.048518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:08.054505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9181598-6454-4f16-8af7-d8c5b348c19f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:08.064727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:08.065830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.066323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.066827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.068490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:08.072160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:08.086545 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1018/2037] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:08.096934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:08.117434 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1020/2038] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:02:08.129373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:08.155390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bf243-6b0d-4244-9916-887025a39883 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:08.162671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ec3ac-b60c-45f1-911d-4c149ef206f9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:08.173766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:08.174695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.175161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.176094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.177099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:08.182525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.191939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.191939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] PATH_INFO: `/v3/projects/17136e1efabf4bcc8d3d15ae323db308/users/e68925e47b56481499e28fc93456b5dd/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.192276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:08.195535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:08.242628 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1021/2039] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:08.242628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:08.245932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:08.265050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] RBAC: Authorizing `identity:create_grant(project_id=17136e1efabf4bcc8d3d15ae323db308, user_id=e68925e47b56481499e28fc93456b5dd, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:08.268266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:08.280287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf52a-018d-4369-acfa-ea6afd8d4b62 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:08.294134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:08.294984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.295296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.295566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] PATH_INFO: `/v3/projects/9984f630cc6e48bbb88d3de67f5fa8f6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.297221 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9984f630cc6e48bbb88d3de67f5fa8f6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:08.299671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:08.307184 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1019/2040] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:08 2022] PUT /identity/v3/projects/17136e1efabf4bcc8d3d15ae323db308/users/e68925e47b56481499e28fc93456b5dd/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:08.308762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:08.431188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:08.465893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9984f630cc6e48bbb88d3de67f5fa8f6', 'request_id': 'req-5be59361-eb4a-4af9-a9fc-795401c68d25'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:08.481704 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9984f630cc6e48bbb88d3de67f5fa8f6', 'request_id': 'req-5be59361-eb4a-4af9-a9fc-795401c68d25'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:08.490917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:08.500114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9984f630cc6e48bbb88d3de67f5fa8f6', 'request_id': 'req-5be59361-eb4a-4af9-a9fc-795401c68d25'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:08.531240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] The token cache is being invalidate because project 9984f630cc6e48bbb88d3de67f5fa8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:08.532311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:08.532874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fb0cf-3674-46e0-a67e-cafca0f3c19b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:08.534120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5be59361-eb4a-4af9-a9fc-795401c68d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:08.537525 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1022/2041] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:08 2022] DELETE /identity/v3/projects/9984f630cc6e48bbb88d3de67f5fa8f6 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:08.540723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee876378-c22c-44c0-a04c-4aa020723c7b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.541992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee876378-c22c-44c0-a04c-4aa020723c7b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.545448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee876378-c22c-44c0-a04c-4aa020723c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.624613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ee876378-c22c-44c0-a04c-4aa020723c7b None None] MFA Rules not processed for user `e68925e47b56481499e28fc93456b5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:08.627812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ee876378-c22c-44c0-a04c-4aa020723c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:08.682701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ce01c-ddf0-4b7f-93e7-96ec4ca928b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:08.684066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce01c-ddf0-4b7f-93e7-96ec4ca928b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.684711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce01c-ddf0-4b7f-93e7-96ec4ca928b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.688079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ce01c-ddf0-4b7f-93e7-96ec4ca928b4 admin admin] PATH_INFO: `/v3/projects/698b4e3ee36843148a62219d277bc561/users/b12534da08af490da82b9ebb9e823741/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.748812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ce01c-ddf0-4b7f-93e7-96ec4ca928b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=698b4e3ee36843148a62219d277bc561, user_id=b12534da08af490da82b9ebb9e823741, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:08.752002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ce01c-ddf0-4b7f-93e7-96ec4ca928b4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:08.773591 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1020/2042] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:08 2022] PUT /identity/v3/projects/698b4e3ee36843148a62219d277bc561/users/b12534da08af490da82b9ebb9e823741/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:08.775252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.775641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.775901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.800678 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1023/2043] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:08 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:08.806414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:08.807879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:08.808192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67b7af-3899-4733-8533-b772ac9eaf15 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.808651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67b7af-3899-4733-8533-b772ac9eaf15 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.809019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67b7af-3899-4733-8533-b772ac9eaf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.868847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ba67b7af-3899-4733-8533-b772ac9eaf15 None None] MFA Rules not processed for user `b12534da08af490da82b9ebb9e823741`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:08.871300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba67b7af-3899-4733-8533-b772ac9eaf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:08.923466 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1021/2044] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:08.924632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:08.937884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:08.956816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981807c-3bc1-487f-99f6-e5b340cbe589 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:08.969126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6cc3a3-1ca0-431c-95dc-5839cc84b3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:08.970959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc3a3-1ca0-431c-95dc-5839cc84b3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:08.973258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc3a3-1ca0-431c-95dc-5839cc84b3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:08.973498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6cc3a3-1ca0-431c-95dc-5839cc84b3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:08.974664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cc3a3-1ca0-431c-95dc-5839cc84b3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:08.979134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6cc3a3-1ca0-431c-95dc-5839cc84b3c7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.045907 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1022/2045] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:08 2022] POST /identity/v3/projects => generated 407 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:09.048976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38f867-6c47-479c-927b-be748928f916 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.049788 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38f867-6c47-479c-927b-be748928f916 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.050385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38f867-6c47-479c-927b-be748928f916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.079306 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1024/2046] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:09.080515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67b7af-3899-4733-8533-b772ac9eaf15 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.109956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67b7af-3899-4733-8533-b772ac9eaf15 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.141209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3f38f867-6c47-479c-927b-be748928f916 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:09.143584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3f38f867-6c47-479c-927b-be748928f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:09.180586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:09.247827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.249070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.251348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.251874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.253136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.258353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.294844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:09.326576 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1025/2047] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:09 2022] POST /identity/v3/users => generated 310 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:09.328584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.331737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:09.346509 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1023/2048] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:09.349453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38f867-6c47-479c-927b-be748928f916 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.376025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38f867-6c47-479c-927b-be748928f916 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.393357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.394411 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.394901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.395756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.396941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.414738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.459437 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1024/2049] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:09.461588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.480233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabfe6-7c6e-4e33-99b7-4dbc755999d9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.491160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.492465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.492995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.493654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.497132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:09.497848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.501560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.537986 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1025/2050] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:09.539194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.566600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3751-89f6-40db-8824-fabd91f3221b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.573956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b5db-79fc-4a2a-8f47-5d5ff5201282 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.588584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.588817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.589764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.589919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.590243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.590355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.590658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] PATH_INFO: `/v3/projects/2c5eb6820fb9434298603b7521017c56/users/818178e4ae76417a9b6530a48706af76/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.592898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.625236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5eb6820fb9434298603b7521017c56, user_id=818178e4ae76417a9b6530a48706af76, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.629015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.662229 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1026/2051] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:09 2022] PUT /identity/v3/projects/2c5eb6820fb9434298603b7521017c56/users/818178e4ae76417a9b6530a48706af76/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:09.663843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.767471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.769230 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.786314 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1026/2052] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:09.788846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.807787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406300ec-db7b-4e3d-b532-14b3b1c5d7b3 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.817985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779bc28-4d51-4045-94a7-a6c409415826 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.818158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.834394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.834394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.834394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.834394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.834394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.835257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.835257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.835257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.835257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.835833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.835833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.836233 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1027/2053] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:09.842797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.867861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f462c82-0713-4e5f-a1b2-2d6e1094f308 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:09.876495 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:09.877973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:09.878301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:09.878781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] PATH_INFO: `/v3/projects/2c5eb6820fb9434298603b7521017c56/users/818178e4ae76417a9b6530a48706af76/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:09.882807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5eb6820fb9434298603b7521017c56, user_id=818178e4ae76417a9b6530a48706af76, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:09.900458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:09.903301 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1027/2054] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:09 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:09.904777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:09.909429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:09.942572 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1028/2055] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:09 2022] PUT /identity/v3/projects/2c5eb6820fb9434298603b7521017c56/users/818178e4ae76417a9b6530a48706af76/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:09.943626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:10.164273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970943-4a11-41cb-978d-bc1485835c8a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:10.221858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:10.234980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aa511-bcc4-4f42-89a1-8ff29c5e05fa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:10.245341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f480d-ce73-4991-9dfa-036d911120e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:10.246543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f480d-ce73-4991-9dfa-036d911120e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:10.247125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f480d-ce73-4991-9dfa-036d911120e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:10.247894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f480d-ce73-4991-9dfa-036d911120e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:10.249162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f480d-ce73-4991-9dfa-036d911120e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:10.254461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f480d-ce73-4991-9dfa-036d911120e7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:10.268576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:10.308974 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1029/2056] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:09 2022] POST /identity/v3/users => generated 293 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:10.312262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:10.313048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:10.313825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:10.333360 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:10.334129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:10.334462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:10.334731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] PATH_INFO: `/v3/users/b12534da08af490da82b9ebb9e823741` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:10.336764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12534da08af490da82b9ebb9e823741)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:10.340739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:10.383501 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] MFA Rules not processed for user `818178e4ae76417a9b6530a48706af76`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:10.388066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:10.483155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12534da08af490da82b9ebb9e823741'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:10.485161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12534da08af490da82b9ebb9e823741'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:10.507520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12534da08af490da82b9ebb9e823741'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:10.546530 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1030/2057] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:10 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:10.547848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12534da08af490da82b9ebb9e823741'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:10.548558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:10.553145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12534da08af490da82b9ebb9e823741'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:10.563103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:10.589712 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1028/2058] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:09 2022] DELETE /identity/v3/users/b12534da08af490da82b9ebb9e823741 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:10.594313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:10.601634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:10.610628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fcca1-eb50-41d6-9ce4-d3575530f0c7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:10.824654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca752c6-07f5-4ca2-8a56-e12f3514a379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:10.825361 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca752c6-07f5-4ca2-8a56-e12f3514a379 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:10.825711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca752c6-07f5-4ca2-8a56-e12f3514a379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:10.826029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca752c6-07f5-4ca2-8a56-e12f3514a379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:10.827009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca752c6-07f5-4ca2-8a56-e12f3514a379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:10.829050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca752c6-07f5-4ca2-8a56-e12f3514a379 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:10.843774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21149a-c793-46a5-b6d5-d435235030a3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:10.854548 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1031/2059] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:10.857342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4e5e7-044d-4cf8-958b-3c3fd24e6f1e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:10.857931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4e5e7-044d-4cf8-958b-3c3fd24e6f1e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:10.858656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4e5e7-044d-4cf8-958b-3c3fd24e6f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:10.859033 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:10.859990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:10.860293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:10.860712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:10.906876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-65e4e5e7-044d-4cf8-958b-3c3fd24e6f1e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:10.910219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-65e4e5e7-044d-4cf8-958b-3c3fd24e6f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:10.933647 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1032/2060] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:10.933647 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4e5e7-044d-4cf8-958b-3c3fd24e6f1e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:10.951987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4e5e7-044d-4cf8-958b-3c3fd24e6f1e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:10.962548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:10.963599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:10.964115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:10.964630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:10.966016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:10.969043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:10.993263 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1033/2061] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:10.995120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:10.998964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:11.051456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:11.056488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:11.085569 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1029/2062] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:10 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:11.087582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:11.104795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:11.118003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db628e47-7b82-4c31-99e4-1e7430b0d3e1 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:11.128757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:11.132072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:11.133025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:11.134042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] PATH_INFO: `/v3/projects/1405355b13794efc95daf56a4d47c3e5/users/7ddf83398cda4df8a0117a7127d2cab1/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:11.137852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb5856-8198-4dd9-aefe-f788f0bec6dc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:11.146419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770906e-91fb-443a-aaab-5a9460e0e61d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:11.159316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770906e-91fb-443a-aaab-5a9460e0e61d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:11.159929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770906e-91fb-443a-aaab-5a9460e0e61d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:11.172601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770906e-91fb-443a-aaab-5a9460e0e61d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:11.184566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1405355b13794efc95daf56a4d47c3e5, user_id=7ddf83398cda4df8a0117a7127d2cab1, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:11.195001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:11.204109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770906e-91fb-443a-aaab-5a9460e0e61d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:11.205689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770906e-91fb-443a-aaab-5a9460e0e61d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:11.209681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:11.224001 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1034/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:11.271464 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1030/2064] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:11 2022] PUT /identity/v3/projects/1405355b13794efc95daf56a4d47c3e5/users/7ddf83398cda4df8a0117a7127d2cab1/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:11.288339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a551e-48a0-4dfb-a6f1-8358230c72d1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:11.288641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a551e-48a0-4dfb-a6f1-8358230c72d1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:11.288991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a551e-48a0-4dfb-a6f1-8358230c72d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:11.432747 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d68a551e-48a0-4dfb-a6f1-8358230c72d1 None None] MFA Rules not processed for user `7ddf83398cda4df8a0117a7127d2cab1`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:11.438414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d68a551e-48a0-4dfb-a6f1-8358230c72d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:11.556703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:11.558916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:11.704901 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1035/2065] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:11.742454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc8309-a144-4901-bb51-044f533d6b2f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:11.744854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a551e-48a0-4dfb-a6f1-8358230c72d1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:11.748438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1636a63-343f-4a91-bcee-a4b7ede55a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:11.749621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636a63-343f-4a91-bcee-a4b7ede55a3a service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:11.750039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636a63-343f-4a91-bcee-a4b7ede55a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:11.750417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1636a63-343f-4a91-bcee-a4b7ede55a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:11.754227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e1636a63-343f-4a91-bcee-a4b7ede55a3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:11.778094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a551e-48a0-4dfb-a6f1-8358230c72d1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:11.807830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:11.809772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:11.810979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:11.811670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:11.832838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:11.837404 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:11.857838 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1036/2066] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:11.896513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1636a63-343f-4a91-bcee-a4b7ede55a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:11.898892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1636a63-343f-4a91-bcee-a4b7ede55a3a service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:11.918556 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1031/2067] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:11 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:12.294743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:12.298759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:12.603845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5df63-08c1-4f11-bcbc-e86a27897349 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:12.627080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:12.628241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:12.633122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:12.633643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] PATH_INFO: `/v3/projects/698b4e3ee36843148a62219d277bc561` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:12.639823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] RBAC: Authorizing `identity:delete_project(project_id=698b4e3ee36843148a62219d277bc561)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:12.645097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:12.824275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698b4e3ee36843148a62219d277bc561', 'request_id': 'req-492746f3-1bd1-467f-be84-5ad76706606d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:12.858734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698b4e3ee36843148a62219d277bc561', 'request_id': 'req-492746f3-1bd1-467f-be84-5ad76706606d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:12.875774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698b4e3ee36843148a62219d277bc561', 'request_id': 'req-492746f3-1bd1-467f-be84-5ad76706606d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:12.901472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:12.916617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] The token cache is being invalidate because project 698b4e3ee36843148a62219d277bc561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:12.918104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:12.926042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-492746f3-1bd1-467f-be84-5ad76706606d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:12.929513 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1037/2068] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:12 2022] DELETE /identity/v3/projects/698b4e3ee36843148a62219d277bc561 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:12.937117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908240-882d-49cd-b372-cec8daea78bf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:12.938328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908240-882d-49cd-b372-cec8daea78bf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:12.938605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15908240-882d-49cd-b372-cec8daea78bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:12.992892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-15908240-882d-49cd-b372-cec8daea78bf None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:12.997724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-15908240-882d-49cd-b372-cec8daea78bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:13.139867 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1032/2069] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:13.155612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49eda5b-f90c-4bba-8abd-906ee758d808 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:13.156561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49eda5b-f90c-4bba-8abd-906ee758d808 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:13.157547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49eda5b-f90c-4bba-8abd-906ee758d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:13.190363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b49eda5b-f90c-4bba-8abd-906ee758d808 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:13.192700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b49eda5b-f90c-4bba-8abd-906ee758d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:13.204243 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1038/2070] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:13.212284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908240-882d-49cd-b372-cec8daea78bf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:13.234887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15908240-882d-49cd-b372-cec8daea78bf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:13.244143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:13.245260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:13.245789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:13.246266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:13.247456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:13.247995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:13.251098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:13.275515 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1033/2071] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:13.283120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49eda5b-f90c-4bba-8abd-906ee758d808 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:13.303661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49eda5b-f90c-4bba-8abd-906ee758d808 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:13.322296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:13.323173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:13.323591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:13.324015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:13.325053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:13.327339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:13.415642 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1039/2072] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:13 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:13.416784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:13.434310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b0664-1eda-4e7f-9a27-76f774bcbe7b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:13.494632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:13.641876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:13.642618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:13.643048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:13.645037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:13.645545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:13.648666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:13.709477 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1034/2073] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:13 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:13.717162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:13.746761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328dfce-d4e6-4f29-9065-aa242422c693 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:13.762439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:13.763488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:13.764087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:13.764537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:13.765895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:13.768933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:13.806069 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1040/2074] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:13.808313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:13.870846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202f123-b132-4212-bcf4-4379d75bcd5d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:13.904109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7423d0-81b6-44fd-8007-e77d80f30c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:13.906254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7423d0-81b6-44fd-8007-e77d80f30c0a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:13.907106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7423d0-81b6-44fd-8007-e77d80f30c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:13.907769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7423d0-81b6-44fd-8007-e77d80f30c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:13.909465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7423d0-81b6-44fd-8007-e77d80f30c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:13.915907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7423d0-81b6-44fd-8007-e77d80f30c0a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:13.953778 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1035/2075] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:13.955177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:13.990789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a362-257d-4f89-97ba-e1b169b9a23d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:14.014798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:14.015491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:14.015765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:14.016034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] PATH_INFO: `/v3/projects/7e60b04bc4b94ede9d400ace2f34e0dd/users/4a07b9d0c9a14995994b00df64a6b851/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:14.057380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e60b04bc4b94ede9d400ace2f34e0dd, user_id=4a07b9d0c9a14995994b00df64a6b851, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:14.070716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:14.141130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09547f3-78b5-4dbf-97b7-2b028604eb1c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:14.141617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09547f3-78b5-4dbf-97b7-2b028604eb1c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:14.141822 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1041/2076] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:13 2022] PUT /identity/v3/projects/7e60b04bc4b94ede9d400ace2f34e0dd/users/4a07b9d0c9a14995994b00df64a6b851/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:14.142086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09547f3-78b5-4dbf-97b7-2b028604eb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:14.208494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d09547f3-78b5-4dbf-97b7-2b028604eb1c None None] MFA Rules not processed for user `4a07b9d0c9a14995994b00df64a6b851`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:14.212096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d09547f3-78b5-4dbf-97b7-2b028604eb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:14.229204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:14.232240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:14.458470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe5cf-a0cd-407b-9977-5612cd99f41d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:14.468277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:14.469918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:14.470518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:14.471694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] PATH_INFO: `/v3/users/1e4901b9773843cbb36d0026a4bd607c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:14.474047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4901b9773843cbb36d0026a4bd607c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:14.478120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:14.521514 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1036/2077] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:14.528666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09547f3-78b5-4dbf-97b7-2b028604eb1c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:14.559480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09547f3-78b5-4dbf-97b7-2b028604eb1c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:14.571453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:14.573046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4901b9773843cbb36d0026a4bd607c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:14.574350 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:14.575087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:14.576088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:14.578466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:14.583722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:14.587779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4901b9773843cbb36d0026a4bd607c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:14.615358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4901b9773843cbb36d0026a4bd607c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:14.617296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4901b9773843cbb36d0026a4bd607c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:14.619185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4b7a7a57-bb70-4df6-bf17-681b8a1df98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4901b9773843cbb36d0026a4bd607c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:14.641620 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1042/2078] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:14 2022] DELETE /identity/v3/users/1e4901b9773843cbb36d0026a4bd607c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:14.650064 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1037/2079] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:14 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:14.656602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:14.657568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:14.658238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:14.666212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:14.675409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:14.688416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:14.690258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:14.714499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:14.859022 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1043/2080] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:14.867938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140cec0-6dad-4344-8f55-81c015490fc0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:14.874037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:14.875091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:14.876146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:14.876529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:14.877180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:14.879478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:14.902869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:14.910378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:14.912816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:14.914581 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1038/2081] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:14 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:14.920474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:14.939925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140f148-9b54-4a05-bc38-ac0f62ecda36 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:14.947662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:14.948760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:14.949383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:14.950301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:14.951414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:14.953539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:14.970094 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1039/2082] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:14.976162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:14.994461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7d376-6c36-4dbd-bf6b-dd8350589d38 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:15.001719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:15.002713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:15.003125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:15.003609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:15.004505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:15.006916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:15.030194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614e44c-3944-499d-932c-d2cc34ed830f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:15.030465 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1040/2083] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:15.032936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:15.041299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996a1d1-92f8-4e14-8b43-9b2381978205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:15.041943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a1d1-92f8-4e14-8b43-9b2381978205 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:15.042219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a1d1-92f8-4e14-8b43-9b2381978205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:15.042491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996a1d1-92f8-4e14-8b43-9b2381978205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:15.060621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc3813-fa0c-42d6-96ac-e5e375b5c98a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:15.067935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996a1d1-92f8-4e14-8b43-9b2381978205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:15.069571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996a1d1-92f8-4e14-8b43-9b2381978205 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:15.069774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:15.070623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:15.071016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:15.071400 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] PATH_INFO: `/v3/projects/b8dce99d49db4484847fa5d7f2f66b79/users/b5599054e3834c17b23e03be7d17fbae/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:15.094985 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1044/2084] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:15.095868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8dce99d49db4484847fa5d7f2f66b79, user_id=b5599054e3834c17b23e03be7d17fbae, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:15.100227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:15.130488 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1041/2085] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:15 2022] PUT /identity/v3/projects/b8dce99d49db4484847fa5d7f2f66b79/users/b5599054e3834c17b23e03be7d17fbae/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:15.150415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d1d3-affd-46fe-b939-30d09c7a3cce None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:15.151081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d1d3-affd-46fe-b939-30d09c7a3cce None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:15.151699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d1d3-affd-46fe-b939-30d09c7a3cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:15.203470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f3a5d1d3-affd-46fe-b939-30d09c7a3cce None None] MFA Rules not processed for user `b5599054e3834c17b23e03be7d17fbae`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:15.205107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f3a5d1d3-affd-46fe-b939-30d09c7a3cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:15.365247 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1045/2086] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:15.369522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:15.537942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f9077-7c8b-4fd0-b19b-fe024c351e00 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:15.550059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:15.551084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:15.551084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:15.551084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:15.551763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:15.554400 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:15.607603 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1042/2087] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:15 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:15.616921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d1d3-affd-46fe-b939-30d09c7a3cce None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:15.634895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d1d3-affd-46fe-b939-30d09c7a3cce None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:15.767684 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:15.786329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:15.788995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:15.830138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:15.831073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:15.831485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:15.831796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:15.832508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:15.837437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc29695-f3ae-43bd-874a-d1370c884365 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:15.839585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:15.849070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:15.850288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:15.851480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:15.852041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] PATH_INFO: `/v3/projects/7a7d872310ec4939b3fbe251dd5e6c24` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:15.854712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7d872310ec4939b3fbe251dd5e6c24)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:15.859328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:15.914343 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1046/2088] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:15 2022] POST /identity/v3/users => generated 298 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:15.915877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:15.962816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64738-6105-4078-bbf4-ef2fdc42f32b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:15.998203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7d872310ec4939b3fbe251dd5e6c24', 'request_id': 'req-580df6da-3ebe-4db5-9af5-ae6e99f7401c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.018143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7d872310ec4939b3fbe251dd5e6c24', 'request_id': 'req-580df6da-3ebe-4db5-9af5-ae6e99f7401c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.027530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7d872310ec4939b3fbe251dd5e6c24', 'request_id': 'req-580df6da-3ebe-4db5-9af5-ae6e99f7401c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.047054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] The token cache is being invalidate because project 7a7d872310ec4939b3fbe251dd5e6c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:16.047375 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.049848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-580df6da-3ebe-4db5-9af5-ae6e99f7401c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.060174 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1043/2089] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:15 2022] DELETE /identity/v3/projects/7a7d872310ec4939b3fbe251dd5e6c24 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:16.063804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7d824-ba7e-421f-8b48-bc2597ea627e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.065491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7d824-ba7e-421f-8b48-bc2597ea627e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.065491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7d824-ba7e-421f-8b48-bc2597ea627e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.095591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eaf7d824-ba7e-421f-8b48-bc2597ea627e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:16.098319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eaf7d824-ba7e-421f-8b48-bc2597ea627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:16.116884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.117686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.118873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.118873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.119293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.123356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:16.123705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.140968 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1047/2090] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:02:16.146531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.148998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:16.199007 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1044/2091] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:16.219273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08856335-57ba-4eef-a3b7-4c1e97a25c9e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.219956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08856335-57ba-4eef-a3b7-4c1e97a25c9e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.222738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08856335-57ba-4eef-a3b7-4c1e97a25c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.229941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fe13d-a359-4f02-b22b-db89822bd8a3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:16.240036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.240819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.241858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.242332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] PATH_INFO: `/v3/projects/88b815e1d5bd4cc8b9c995d5cded0ed7/users/4cedd6c1d80740fb84b7460f18ef9846/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.274291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-08856335-57ba-4eef-a3b7-4c1e97a25c9e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:16.276058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-08856335-57ba-4eef-a3b7-4c1e97a25c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:16.292005 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1045/2092] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:16.296839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08856335-57ba-4eef-a3b7-4c1e97a25c9e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.311062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b815e1d5bd4cc8b9c995d5cded0ed7, user_id=4cedd6c1d80740fb84b7460f18ef9846, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.314355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.319938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08856335-57ba-4eef-a3b7-4c1e97a25c9e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:16.347114 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1048/2093] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:16 2022] PUT /identity/v3/projects/88b815e1d5bd4cc8b9c995d5cded0ed7/users/4cedd6c1d80740fb84b7460f18ef9846/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:16.349817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.350676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.351494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.352759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.354255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.357266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:16.361340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.364659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.401884 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1046/2094] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:16.419790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.454670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6fd3c-fbe5-4fb6-bc45-887059e75ad8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:16.496328 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ac56f-02f9-47cc-a97c-54323e7807bd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:16.507854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.509383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.510613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.511125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.512822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.516367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.570025 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1049/2095] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:16.573943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.574206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.574597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.575006 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.575416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.576248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.579473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.613318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5c55a-a12b-410d-8156-f84cbd7201ee admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:16.620569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.621627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.621627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.621924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] PATH_INFO: `/v3/projects/88b815e1d5bd4cc8b9c995d5cded0ed7/users/4cedd6c1d80740fb84b7460f18ef9846/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.624872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b815e1d5bd4cc8b9c995d5cded0ed7, user_id=4cedd6c1d80740fb84b7460f18ef9846, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.628554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.643627 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1047/2096] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:16 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:16.644904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.646815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:16.647025 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1050/2097] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:16 2022] PUT /identity/v3/projects/88b815e1d5bd4cc8b9c995d5cded0ed7/users/4cedd6c1d80740fb84b7460f18ef9846/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:16.657863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.715448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:16.772710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25356ab7-acce-4c55-b6c7-d266320aaa79 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:16.801793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0563c-0165-424e-8f38-dff857274871 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:16.811497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.812757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.813486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.813862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] PATH_INFO: `/v3/users/7ddf83398cda4df8a0117a7127d2cab1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.817030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddf83398cda4df8a0117a7127d2cab1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.821038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.867310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:16.883391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1977ba0-1447-4ec0-9679-ef30239d8a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:16.884551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1977ba0-1447-4ec0-9679-ef30239d8a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.885462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1977ba0-1447-4ec0-9679-ef30239d8a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.885462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1977ba0-1447-4ec0-9679-ef30239d8a06 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.886444 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1977ba0-1447-4ec0-9679-ef30239d8a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:16.890805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1977ba0-1447-4ec0-9679-ef30239d8a06 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:16.912703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddf83398cda4df8a0117a7127d2cab1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.914420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddf83398cda4df8a0117a7127d2cab1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.924433 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1051/2098] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:16 2022] POST /identity/v3/users => generated 288 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:16.925916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddf83398cda4df8a0117a7127d2cab1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.929750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645decf-0c72-4fec-a5b4-e49d3755e7d9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:16.930134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645decf-0c72-4fec-a5b4-e49d3755e7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:16.930516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645decf-0c72-4fec-a5b4-e49d3755e7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:16.951421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddf83398cda4df8a0117a7127d2cab1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.952736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddf83398cda4df8a0117a7127d2cab1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:16.965288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7645decf-0c72-4fec-a5b4-e49d3755e7d9 None None] MFA Rules not processed for user `4cedd6c1d80740fb84b7460f18ef9846`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:16.966728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7645decf-0c72-4fec-a5b4-e49d3755e7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:16.981163 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1048/2099] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:16 2022] DELETE /identity/v3/users/7ddf83398cda4df8a0117a7127d2cab1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:16.982315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:16.983930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:17.101492 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1052/2100] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:17.102539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621c59-022d-4e05-bec9-d0e4c9f31548 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:17.104773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeba87-322b-462f-b0ab-a065505532fd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:17.105215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeba87-322b-462f-b0ab-a065505532fd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:17.105733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeba87-322b-462f-b0ab-a065505532fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:17.114182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:17.115158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:17.115540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:17.115921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:17.117862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:17.121135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:17.135132 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1049/2101] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:17.136039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:17.138339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:17.143039 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-48eeba87-322b-462f-b0ab-a065505532fd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:17.145235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-48eeba87-322b-462f-b0ab-a065505532fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:17.159935 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1053/2102] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:17.161032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eeba87-322b-462f-b0ab-a065505532fd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:17.173654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eeba87-322b-462f-b0ab-a065505532fd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:17.181542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:17.182190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:17.182472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:17.182744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:17.183454 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:17.185379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:17.197583 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1054/2103] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:17.198564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:17.200608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:17.238942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780b4b0-32bf-4724-99c2-8290081488f6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:17.244879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abace7-dabb-4a93-a258-0bfda1ee95de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:17.245518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abace7-dabb-4a93-a258-0bfda1ee95de service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:17.245799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abace7-dabb-4a93-a258-0bfda1ee95de service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:17.246075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abace7-dabb-4a93-a258-0bfda1ee95de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:17.248094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e6abace7-dabb-4a93-a258-0bfda1ee95de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:17.275480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:17.305719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ef06d-e1c2-4f20-8fbe-d8e953f15448 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:17.316716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:17.318025 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:17.318345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:17.318701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:17.331154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:17.333953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:17.356344 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1055/2104] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:17.357305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:17.360741 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:17.366638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abace7-dabb-4a93-a258-0bfda1ee95de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:17.368898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abace7-dabb-4a93-a258-0bfda1ee95de service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:17.373089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384038da-0e9c-405d-b524-06b178a0ca89 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:17.382634 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1050/2105] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:17.383174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:17.384322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:17.384852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:17.385180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] PATH_INFO: `/v3/projects/33642e8ad0c3498ab62b99d210fdf560/users/a37ef63fceda47a8863cdab1b904c2be/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:17.422026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=33642e8ad0c3498ab62b99d210fdf560, user_id=a37ef63fceda47a8863cdab1b904c2be, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:17.427960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:17.465133 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1056/2106] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:17 2022] PUT /identity/v3/projects/33642e8ad0c3498ab62b99d210fdf560/users/a37ef63fceda47a8863cdab1b904c2be/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:17.484354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:17.484654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:17.485114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:17.568315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] MFA Rules not processed for user `a37ef63fceda47a8863cdab1b904c2be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:17.572059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:17.910401 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1051/2107] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:17.954977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:18.139835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e3947-8c67-474a-87e5-b60d9c868c1d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:18.149325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ee5b3-246f-4e86-8757-ab1c33b8eec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:18.150323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ee5b3-246f-4e86-8757-ab1c33b8eec5 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:18.151262 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ee5b3-246f-4e86-8757-ab1c33b8eec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:18.151821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ee5b3-246f-4e86-8757-ab1c33b8eec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:18.189290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ee5b3-246f-4e86-8757-ab1c33b8eec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:18.191453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ee5b3-246f-4e86-8757-ab1c33b8eec5 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:18.208501 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1057/2108] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:18.580625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:18.583421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:18.808837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94599f9-6746-4133-8eac-e267dd6a0c14 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:18.815625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:18.816496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:18.816897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:18.817212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] PATH_INFO: `/v3/projects/1405355b13794efc95daf56a4d47c3e5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:18.818970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1405355b13794efc95daf56a4d47c3e5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:18.821447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:18.984971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1405355b13794efc95daf56a4d47c3e5', 'request_id': 'req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.000581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1405355b13794efc95daf56a4d47c3e5', 'request_id': 'req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.012560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1405355b13794efc95daf56a4d47c3e5', 'request_id': 'req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.036551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] The token cache is being invalidate because project 1405355b13794efc95daf56a4d47c3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:19.037023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.039383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0d2cc29-e14e-43b3-8b33-4afdd13ac0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.041948 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1052/2109] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:18 2022] DELETE /identity/v3/projects/1405355b13794efc95daf56a4d47c3e5 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:19.050983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564676-5eae-4fd7-91a0-1dc5bb456948 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:19.051671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564676-5eae-4fd7-91a0-1dc5bb456948 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:19.051824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62564676-5eae-4fd7-91a0-1dc5bb456948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:19.088560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-62564676-5eae-4fd7-91a0-1dc5bb456948 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:19.091622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62564676-5eae-4fd7-91a0-1dc5bb456948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:19.210877 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1058/2110] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:19.226125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73769-5544-4019-a866-7f1e277a6bea None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:19.226425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73769-5544-4019-a866-7f1e277a6bea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:19.226728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d73769-5544-4019-a866-7f1e277a6bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:19.268461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-18d73769-5544-4019-a866-7f1e277a6bea None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:19.269725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-18d73769-5544-4019-a866-7f1e277a6bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:19.290115 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1053/2111] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:19.291978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564676-5eae-4fd7-91a0-1dc5bb456948 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:19.307853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62564676-5eae-4fd7-91a0-1dc5bb456948 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:19.316460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:19.317964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:19.318550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:19.319026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:19.320678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:19.321106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:19.323745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:19.355189 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1059/2112] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:19.356437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73769-5544-4019-a866-7f1e277a6bea None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:19.372586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d73769-5544-4019-a866-7f1e277a6bea None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:19.377960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:19.378670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:19.378994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:19.379268 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:19.379932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:19.382182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:19.388894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:19.431953 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1054/2113] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:19 2022] POST /identity/v3/projects => generated 391 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:19.438204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:19.455080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f33c5-d425-4294-b27a-0f58bd301bf4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:19.465476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:19.470090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:19.607090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:19.646856 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:19.648935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a95da7-66f2-4069-93df-62f2d59d2542 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:19.651365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:19.651969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:19.652512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:19.653942 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:19.655762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:19.656547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:19.657144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:19.657600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] PATH_INFO: `/v3/users/e68925e47b56481499e28fc93456b5dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:19.658433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:19.659139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68925e47b56481499e28fc93456b5dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:19.661396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:19.723066 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1060/2114] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:19 2022] POST /identity/v3/users => generated 296 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:19.724468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:19.728137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:19.767633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68925e47b56481499e28fc93456b5dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.768835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68925e47b56481499e28fc93456b5dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.781405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68925e47b56481499e28fc93456b5dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.802870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68925e47b56481499e28fc93456b5dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.804323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68925e47b56481499e28fc93456b5dd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:19.841525 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1055/2115] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:19 2022] DELETE /identity/v3/users/e68925e47b56481499e28fc93456b5dd => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:19.842752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:19.844569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:19.879716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f4385-466f-47b5-9f6a-852f43637778 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:20.007477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063f89a-d3ce-4660-9364-a367bb9d8ec6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:20.018470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea921f-24f7-4eb7-8d6f-a94d35c44f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:20.019551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea921f-24f7-4eb7-8d6f-a94d35c44f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:20.020015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea921f-24f7-4eb7-8d6f-a94d35c44f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:20.020515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea921f-24f7-4eb7-8d6f-a94d35c44f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:20.022109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea921f-24f7-4eb7-8d6f-a94d35c44f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:20.026291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea921f-24f7-4eb7-8d6f-a94d35c44f12 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:20.040082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:20.053178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:20.054692 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:20.056142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:20.059538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:20.084804 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1056/2116] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:20.092962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6a52-123d-4288-a327-a12597e4a3d0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:20.093262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6a52-123d-4288-a327-a12597e4a3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:20.093519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6a52-123d-4288-a327-a12597e4a3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:20.125166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:20.162816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b5ec6a52-123d-4288-a327-a12597e4a3d0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:20.168297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5ec6a52-123d-4288-a327-a12597e4a3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:20.199077 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1057/2117] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:20.200675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec6a52-123d-4288-a327-a12597e4a3d0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:20.220717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec6a52-123d-4288-a327-a12597e4a3d0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:20.232097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:20.232097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:20.232097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:20.232097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:20.232900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:20.232900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:20.254125 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1058/2118] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:20.264891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:20.303027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2287893-325e-4001-9c26-6e055cd060f6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:20.317056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:20.320682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:20.328601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf7013-47bf-4356-8cee-153546d19eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:20.331946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf7013-47bf-4356-8cee-153546d19eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:20.333314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf7013-47bf-4356-8cee-153546d19eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:20.335596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf7013-47bf-4356-8cee-153546d19eb5 admin admin] PATH_INFO: `/v3/projects/bb6109a0a7754f4f9771b1bd403e248d/users/1274140873d84281800293848f480c0b/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:20.349642 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1061/2119] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:20.351120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:20.353090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:20.378473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf7013-47bf-4356-8cee-153546d19eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6109a0a7754f4f9771b1bd403e248d, user_id=1274140873d84281800293848f480c0b, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:20.385243 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf7013-47bf-4356-8cee-153546d19eb5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:20.422476 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1059/2120] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:20 2022] PUT /identity/v3/projects/bb6109a0a7754f4f9771b1bd403e248d/users/1274140873d84281800293848f480c0b/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:20.434620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:20.435094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:20.435558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:20.487231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:20.497419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] MFA Rules not processed for user `1274140873d84281800293848f480c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:20.500686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:20.507765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4706a3-4dca-4223-b806-beecafd55a18 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:20.652209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:20.653145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:20.653728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:20.654024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:20.673075 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1060/2121] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:20.802927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:20.807152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:20.864274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:20.867544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:20.929265 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1062/2122] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:21.002594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf01d59-3c2e-478b-902d-2144f107a708 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:21.018368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:21.019442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:21.020967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:21.021497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:21.055610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:21.059540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:21.084213 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1061/2123] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:21.822862 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:21.823779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:21.862584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8c832-4cd3-4d77-8e94-675686fb3671 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:21.878505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:21.879353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:21.879600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:21.879885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] PATH_INFO: `/v3/projects/17136e1efabf4bcc8d3d15ae323db308` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:21.883451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17136e1efabf4bcc8d3d15ae323db308)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:21.886281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:22.076426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17136e1efabf4bcc8d3d15ae323db308', 'request_id': 'req-4a450811-9e8c-4b9c-9f57-78956b457f2a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.096195 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17136e1efabf4bcc8d3d15ae323db308', 'request_id': 'req-4a450811-9e8c-4b9c-9f57-78956b457f2a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.123968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17136e1efabf4bcc8d3d15ae323db308', 'request_id': 'req-4a450811-9e8c-4b9c-9f57-78956b457f2a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.172705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] The token cache is being invalidate because project 17136e1efabf4bcc8d3d15ae323db308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:22.173098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.175054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.180876 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1063/2124] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:21 2022] DELETE /identity/v3/projects/17136e1efabf4bcc8d3d15ae323db308 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:22.187971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:22.224347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab2efe-021f-42a5-befe-ca193f1194d0 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:22.275536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:22.278203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:22.367564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:22.383397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:22.383715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:22.384147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:22.384532 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] PATH_INFO: `/v3/users/818178e4ae76417a9b6530a48706af76` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:22.386622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=818178e4ae76417a9b6530a48706af76)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:22.390508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:22.458197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:22.565102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a450811-9e8c-4b9c-9f57-78956b457f2a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:22.577272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:22.578362 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:22.580772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:22.582335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:22.587529 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:22.610597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818178e4ae76417a9b6530a48706af76'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.612749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818178e4ae76417a9b6530a48706af76'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.633418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818178e4ae76417a9b6530a48706af76'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.685608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818178e4ae76417a9b6530a48706af76'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.687736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818178e4ae76417a9b6530a48706af76'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:22.726860 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1062/2125] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:22 2022] DELETE /identity/v3/users/818178e4ae76417a9b6530a48706af76 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:22.786383 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:22.790761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:22.828231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:22.830894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:22.833006 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1064/2126] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:23.044270 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b7d17-9ead-4bb5-afd1-3829f88fd4cb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:23.051679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:23.052464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:23.052748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:23.053635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] PATH_INFO: `/v3/users/ef66ca98e383463994017e057c499c6f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:23.055275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef66ca98e383463994017e057c499c6f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:23.057559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:23.146831 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef66ca98e383463994017e057c499c6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.148729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef66ca98e383463994017e057c499c6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.163340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef66ca98e383463994017e057c499c6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.181076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef66ca98e383463994017e057c499c6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.181874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef66ca98e383463994017e057c499c6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.184202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:23.205740 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1063/2127] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:22 2022] DELETE /identity/v3/users/ef66ca98e383463994017e057c499c6f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:23.355931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:23.357769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:23.391095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6f0fe-2eb8-49ca-9ea8-8b160affd699 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:23.541613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:23.542275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:23.543690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:23.544162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] PATH_INFO: `/v3/projects/2c5eb6820fb9434298603b7521017c56` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:23.547004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5eb6820fb9434298603b7521017c56)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:23.551388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:23.622912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:23.624512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:23.653260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd1f5-bd7f-4e05-8dda-8760acc56e13 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:23.653835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5eb6820fb9434298603b7521017c56', 'request_id': 'req-9b353a73-c1af-4cde-9217-2d8c7a76cf65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.667707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5eb6820fb9434298603b7521017c56', 'request_id': 'req-9b353a73-c1af-4cde-9217-2d8c7a76cf65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.680313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5eb6820fb9434298603b7521017c56', 'request_id': 'req-9b353a73-c1af-4cde-9217-2d8c7a76cf65'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.698448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] The token cache is being invalidate because project 2c5eb6820fb9434298603b7521017c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:23.699866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.701266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9b353a73-c1af-4cde-9217-2d8c7a76cf65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.704280 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1065/2128] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:23 2022] DELETE /identity/v3/projects/2c5eb6820fb9434298603b7521017c56 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:23.715582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70b15d-42e8-4ca9-aa4a-27dcaeb312a2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:23.716054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70b15d-42e8-4ca9-aa4a-27dcaeb312a2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:23.716545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70b15d-42e8-4ca9-aa4a-27dcaeb312a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:23.748621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eb70b15d-42e8-4ca9-aa4a-27dcaeb312a2 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:23.750104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eb70b15d-42e8-4ca9-aa4a-27dcaeb312a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:23.764355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:23.772970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:23.774101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:23.774571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:23.776228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] PATH_INFO: `/v3/projects/0b369e7845314685b4145d3feaa02258` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:23.778074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b369e7845314685b4145d3feaa02258)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:23.780536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:23.841947 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1066/2129] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:23.851223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0249b48-5a36-4f6a-9f9a-4b3dd1c121c0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:23.851715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0249b48-5a36-4f6a-9f9a-4b3dd1c121c0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:23.852198 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0249b48-5a36-4f6a-9f9a-4b3dd1c121c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:23.867482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b369e7845314685b4145d3feaa02258', 'request_id': 'req-2d654760-4bec-440c-95a3-047c37aeac0d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.880745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b369e7845314685b4145d3feaa02258', 'request_id': 'req-2d654760-4bec-440c-95a3-047c37aeac0d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.888246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e0249b48-5a36-4f6a-9f9a-4b3dd1c121c0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:23.888625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b369e7845314685b4145d3feaa02258', 'request_id': 'req-2d654760-4bec-440c-95a3-047c37aeac0d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.890452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e0249b48-5a36-4f6a-9f9a-4b3dd1c121c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:23.904221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] The token cache is being invalidate because project 0b369e7845314685b4145d3feaa02258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:23.904739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.905635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d654760-4bec-440c-95a3-047c37aeac0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:23.907611 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1064/2130] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:23 2022] DELETE /identity/v3/projects/0b369e7845314685b4145d3feaa02258 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:23.913799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bd04-dddc-4637-a846-09700ef00da0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:23.914069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bd04-dddc-4637-a846-09700ef00da0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:23.914365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06bd04-dddc-4637-a846-09700ef00da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:23.944064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bb06bd04-dddc-4637-a846-09700ef00da0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:23.946427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bb06bd04-dddc-4637-a846-09700ef00da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:23.999765 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1067/2131] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:24.007951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0249b48-5a36-4f6a-9f9a-4b3dd1c121c0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.019245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0249b48-5a36-4f6a-9f9a-4b3dd1c121c0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.029548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.030397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.030827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.031124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.031865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:24.032614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.034288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.052536 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1068/2132] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:24.058708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.076779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e348a8a-acfd-4f4f-8594-a45f27ac0a67 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.088058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.088995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.089914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.090159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.091330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.093379 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1065/2133] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:24.094044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.109525 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b60c6d-fa08-43f5-bc34-548fd2fb2ef1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.110054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b60c6d-fa08-43f5-bc34-548fd2fb2ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.110506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b60c6d-fa08-43f5-bc34-548fd2fb2ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.155309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-54b60c6d-fa08-43f5-bc34-548fd2fb2ef1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:24.157366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-54b60c6d-fa08-43f5-bc34-548fd2fb2ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:24.174540 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1066/2134] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:24.180575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b60c6d-fa08-43f5-bc34-548fd2fb2ef1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.190986 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1069/2135] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:24 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:24.192350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.208179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b60c6d-fa08-43f5-bc34-548fd2fb2ef1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.234693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:24.355782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa7a29-809d-4e26-a880-f5cd710ab340 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.356595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.358654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.359385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.360052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.361450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:24.362183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.364138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.365299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.365772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.366265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.367205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.367472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.371381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.406363 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1067/2136] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:24.410280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.434008 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1070/2137] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:24 2022] POST /identity/v3/users => generated 284 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:24.434827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbcdb8-63d0-400c-86ea-88bfb5e53ee4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.443159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.450100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.451155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.451911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.453060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.458490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.464272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.465909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b263d57-061d-4ea9-b876-e8868a39470f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.474701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.475789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.476257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.477651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.478410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.487113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.497529 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:24.512738 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1071/2138] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:24.515861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.538207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c18f57-8f17-468c-b2cf-588654eaa5a6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.546179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.547378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.547914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.548273 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.549492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.556286 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1068/2139] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:24 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:24.556568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.557273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.579409 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1072/2140] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:24.580721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.603550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ed5ec-c4b3-49ca-8a4f-3196b97b2208 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.739594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539510be-9465-4e67-b218-ba9f9267c967 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.755625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.757147 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.757754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.758321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.761092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.765541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.821545 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1069/2141] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:24 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:24.833216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.879343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fbb55-ba13-4244-8b64-b5015332ce54 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:24.893402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acce4f-b8ec-40bc-ad0f-b38665cd8cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.894481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acce4f-b8ec-40bc-ad0f-b38665cd8cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.894954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acce4f-b8ec-40bc-ad0f-b38665cd8cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.895409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acce4f-b8ec-40bc-ad0f-b38665cd8cf3 admin admin] PATH_INFO: `/v3/projects/418681cc67234a4aa44f3f4d4f765fc8/users/008ac89bb1bc4eb787a710818df779c5/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.913615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:24.913615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:24.913615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:24.913615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:24.914077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.914077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.962318 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1070/2142] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:24.966662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:24.968287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acce4f-b8ec-40bc-ad0f-b38665cd8cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=418681cc67234a4aa44f3f4d4f765fc8, user_id=008ac89bb1bc4eb787a710818df779c5, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:24.972352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acce4f-b8ec-40bc-ad0f-b38665cd8cf3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:24.993174 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a5400-5e39-4ab0-83de-591861f0a85e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.005586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:25.007460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.008179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.008942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.009809 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1073/2143] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:24 2022] PUT /identity/v3/projects/418681cc67234a4aa44f3f4d4f765fc8/users/008ac89bb1bc4eb787a710818df779c5/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:25.021316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:25.021316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:25.024024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3fe70-a907-496e-a71b-1e572b5e13ea None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.024303 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3fe70-a907-496e-a71b-1e572b5e13ea None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.024777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3fe70-a907-496e-a71b-1e572b5e13ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.030234 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1071/2144] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:25.039985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:25.067074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230a2cf-d461-479e-9f7e-f2eb518fd997 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.087068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6db3fe70-a907-496e-a71b-1e572b5e13ea None None] MFA Rules not processed for user `008ac89bb1bc4eb787a710818df779c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:25.090686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6db3fe70-a907-496e-a71b-1e572b5e13ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:25.182334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:25.200170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95eefc0-2bbb-400f-af3f-12143851e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:25.200954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eefc0-2bbb-400f-af3f-12143851e54d admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.201496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eefc0-2bbb-400f-af3f-12143851e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.201977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eefc0-2bbb-400f-af3f-12143851e54d admin admin] PATH_INFO: `/v3/projects/b388fc6841ed4e5598fba4e8856b7af5/users/9d73ba36238f449dbe08d8888920510f/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.230615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95eefc0-2bbb-400f-af3f-12143851e54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b388fc6841ed4e5598fba4e8856b7af5, user_id=9d73ba36238f449dbe08d8888920510f, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:25.232725 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1074/2145] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:25.234822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95eefc0-2bbb-400f-af3f-12143851e54d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:25.236863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3fe70-a907-496e-a71b-1e572b5e13ea None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:25.255837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3fe70-a907-496e-a71b-1e572b5e13ea None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.266452 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1072/2146] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:25 2022] PUT /identity/v3/projects/b388fc6841ed4e5598fba4e8856b7af5/users/9d73ba36238f449dbe08d8888920510f/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:25.284895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.284895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.284895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.301189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:25.302621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:25.381001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:25.381571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.381909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.382229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.382947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:25.385633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:25.395378 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1073/2147] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:25.405022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:25.413926 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:25.417117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:25.447111 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1075/2148] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:25 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:25.454136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:25.556104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be08972-1f63-458b-b96d-09134f9721d3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.589443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee6e9-3bcc-4695-8963-7da1f24e8e5b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.599343 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:25.600370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.601390 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.601847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.602943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:25.607489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:25.649666 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1076/2149] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:25 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:25.654352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:25.664854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:25.666515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.667479 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.667928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.674150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df249-a5df-4c29-b434-0fd64c75a226 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.680540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:25.681603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.682116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.682690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.683701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:25.686115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:25.688650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:25.691726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:25.704269 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1077/2150] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:02:25.707105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:25.712143 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1074/2151] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:25.728427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e038b-cb55-4f70-bd37-4939b85031b3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.737723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:25.738905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:25.739420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:25.739927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] PATH_INFO: `/v3/projects/738b4076128a4b4c82061d193750e5b3/users/e98fb2afe1844db5b528d03eea0fef50/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:25.779152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=738b4076128a4b4c82061d193750e5b3, user_id=e98fb2afe1844db5b528d03eea0fef50, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:25.784933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:25.816072 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1078/2152] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:25 2022] PUT /identity/v3/projects/738b4076128a4b4c82061d193750e5b3/users/e98fb2afe1844db5b528d03eea0fef50/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:25.822305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:25.844121 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a2242-070b-40a8-97b2-adfca120c973 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:25.927254 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:26.057971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a08c74-dbe1-40c6-ba34-eee7d954a933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:26.059453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08c74-dbe1-40c6-ba34-eee7d954a933 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:26.060714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08c74-dbe1-40c6-ba34-eee7d954a933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:26.061819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a08c74-dbe1-40c6-ba34-eee7d954a933 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:26.064001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a08c74-dbe1-40c6-ba34-eee7d954a933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:26.073545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a08c74-dbe1-40c6-ba34-eee7d954a933 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:26.134796 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1075/2153] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:26.140331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:26.181889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f16b-5cf4-46ed-8b35-b67945138e6d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:26.204492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:26.207503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:26.209506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:26.211571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] PATH_INFO: `/v3/projects/738b4076128a4b4c82061d193750e5b3/users/e98fb2afe1844db5b528d03eea0fef50/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:26.216555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738b4076128a4b4c82061d193750e5b3, user_id=e98fb2afe1844db5b528d03eea0fef50, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:26.224139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:26.248096 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1079/2154] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:26 2022] PUT /identity/v3/projects/738b4076128a4b4c82061d193750e5b3/users/e98fb2afe1844db5b528d03eea0fef50/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:26.263758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d57806-19e6-495d-b936-c54c767f6142 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:26.264580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d57806-19e6-495d-b936-c54c767f6142 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:26.264901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d57806-19e6-495d-b936-c54c767f6142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:26.380450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-16d57806-19e6-495d-b936-c54c767f6142 None None] MFA Rules not processed for user `e98fb2afe1844db5b528d03eea0fef50`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:26.384404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-16d57806-19e6-495d-b936-c54c767f6142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:26.611295 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1076/2155] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:26.750257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:26.752546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:26.879871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b068d-7494-4f04-9e22-2849ca2cfa33 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:26.888086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:26.889152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:26.889835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:26.889835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:26.905628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:26.906747 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:26.951266 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1080/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:27.267436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d57806-19e6-495d-b936-c54c767f6142 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:27.292524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d57806-19e6-495d-b936-c54c767f6142 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:27.297213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e4517-5d8a-41f3-8d45-d04e479a5077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:27.297837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e4517-5d8a-41f3-8d45-d04e479a5077 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:27.298114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e4517-5d8a-41f3-8d45-d04e479a5077 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:27.298542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e4517-5d8a-41f3-8d45-d04e479a5077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:27.457554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e4517-5d8a-41f3-8d45-d04e479a5077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:27.461858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e4517-5d8a-41f3-8d45-d04e479a5077 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:27.489380 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1077/2157] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:27.982342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:27.986127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:28.155660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:28.227040 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a2c53-2d13-4e09-8a1c-8504ef635200 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:28.248998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:28.250516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:28.266216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:28.272232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] PATH_INFO: `/v3/users/4a07b9d0c9a14995994b00df64a6b851` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:28.277447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a07b9d0c9a14995994b00df64a6b851)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:28.283924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:28.491037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a07b9d0c9a14995994b00df64a6b851'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:28.514035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a07b9d0c9a14995994b00df64a6b851'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:28.548107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a07b9d0c9a14995994b00df64a6b851'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:28.549358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a07b9d0c9a14995994b00df64a6b851'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:28.551100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a07b9d0c9a14995994b00df64a6b851'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:28.590181 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1081/2158] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:27 2022] DELETE /identity/v3/users/4a07b9d0c9a14995994b00df64a6b851 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:28.592510 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:28.594120 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:28.595148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:28.660299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:28.668488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:28.832798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:28.849625 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1078/2159] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:28.878857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:28.881585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:29.069508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56113110-f168-4fda-a74f-da94bc544a45 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:29.083670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:29.083670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:29.084043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:29.084222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:29.130557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:29.138125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:29.188314 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1082/2160] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:30.203694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:30.206315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:30.302968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c7e18-6799-4784-8f55-e4021799f9b0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:30.332579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:30.336229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:30.351429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:30.353945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] PATH_INFO: `/v3/projects/7e60b04bc4b94ede9d400ace2f34e0dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:30.361399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e60b04bc4b94ede9d400ace2f34e0dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:30.375674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:30.614150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e60b04bc4b94ede9d400ace2f34e0dd', 'request_id': 'req-e335c778-d01f-48b6-bb6a-90ed687f8f9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:30.661022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e60b04bc4b94ede9d400ace2f34e0dd', 'request_id': 'req-e335c778-d01f-48b6-bb6a-90ed687f8f9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:30.675487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e60b04bc4b94ede9d400ace2f34e0dd', 'request_id': 'req-e335c778-d01f-48b6-bb6a-90ed687f8f9a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:30.750475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] The token cache is being invalidate because project 7e60b04bc4b94ede9d400ace2f34e0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:30.751134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:30.753119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:30.759154 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1079/2161] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:30 2022] DELETE /identity/v3/projects/7e60b04bc4b94ede9d400ace2f34e0dd => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:30.775218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:30.815386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b53e0a-9ed0-4062-bb1b-a6b78a764d68 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:31.011965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:31.055588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:31.057859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:31.058253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:31.058756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] PATH_INFO: `/v3/users/b5599054e3834c17b23e03be7d17fbae` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:31.078107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5599054e3834c17b23e03be7d17fbae)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:31.083450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:31.284581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5599054e3834c17b23e03be7d17fbae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:31.314849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5599054e3834c17b23e03be7d17fbae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:31.361835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5599054e3834c17b23e03be7d17fbae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:31.363190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5599054e3834c17b23e03be7d17fbae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:31.364481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5599054e3834c17b23e03be7d17fbae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:31.399323 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1083/2162] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:30 2022] DELETE /identity/v3/users/b5599054e3834c17b23e03be7d17fbae => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:32.902984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:32.908908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:32.940228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335c778-d01f-48b6-bb6a-90ed687f8f9a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:33.123092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:33.123937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:33.124533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:33.124968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] PATH_INFO: `/v3/projects/b8dce99d49db4484847fa5d7f2f66b79` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:33.126267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8dce99d49db4484847fa5d7f2f66b79)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:33.128355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:33.223141 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8dce99d49db4484847fa5d7f2f66b79', 'request_id': 'req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.235873 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8dce99d49db4484847fa5d7f2f66b79', 'request_id': 'req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.250118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8dce99d49db4484847fa5d7f2f66b79', 'request_id': 'req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.272999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] The token cache is being invalidate because project b8dce99d49db4484847fa5d7f2f66b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:33.273727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.275209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.278463 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1080/2163] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:32 2022] DELETE /identity/v3/projects/b8dce99d49db4484847fa5d7f2f66b79 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:33.284263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:33.286580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:33.313917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a66d-f3e6-4b2d-9649-b19aa40bf847 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:33.432773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:33.433656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:33.434555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:33.434843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] PATH_INFO: `/v3/users/4cedd6c1d80740fb84b7460f18ef9846` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:33.437070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cedd6c1d80740fb84b7460f18ef9846)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:33.441391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:33.529787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cedd6c1d80740fb84b7460f18ef9846'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.543960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cedd6c1d80740fb84b7460f18ef9846'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.565822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cedd6c1d80740fb84b7460f18ef9846'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.567136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cedd6c1d80740fb84b7460f18ef9846'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.569453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cedd6c1d80740fb84b7460f18ef9846'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:33.577966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:33.610167 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1084/2164] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:33 2022] DELETE /identity/v3/users/4cedd6c1d80740fb84b7460f18ef9846 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:34.172144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:34.175849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:34.231283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f6b6-8613-439c-96b7-76f8f7dfc2e2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:34.258575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:34.388921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:34.393808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:34.438869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:34.440012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:34.440578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:34.441103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] PATH_INFO: `/v3/projects/88b815e1d5bd4cc8b9c995d5cded0ed7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:34.443281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b815e1d5bd4cc8b9c995d5cded0ed7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:34.448480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:34.528620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b815e1d5bd4cc8b9c995d5cded0ed7', 'request_id': 'req-949e7cec-c73e-470b-b095-ab5ba998bb17'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.545878 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b815e1d5bd4cc8b9c995d5cded0ed7', 'request_id': 'req-949e7cec-c73e-470b-b095-ab5ba998bb17'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.558080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b815e1d5bd4cc8b9c995d5cded0ed7', 'request_id': 'req-949e7cec-c73e-470b-b095-ab5ba998bb17'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.590877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] The token cache is being invalidate because project 88b815e1d5bd4cc8b9c995d5cded0ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:34.591163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7528d3-575e-47db-94fc-a97469fdb08e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:34.592449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.593790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.596685 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1081/2165] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:34 2022] DELETE /identity/v3/projects/88b815e1d5bd4cc8b9c995d5cded0ed7 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:34.598866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:34.607431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:34.687814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:34.688489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:34.688964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:34.689239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] PATH_INFO: `/v3/users/a37ef63fceda47a8863cdab1b904c2be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:34.690723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37ef63fceda47a8863cdab1b904c2be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:34.693575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:34.791665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37ef63fceda47a8863cdab1b904c2be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.807658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37ef63fceda47a8863cdab1b904c2be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.844837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37ef63fceda47a8863cdab1b904c2be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.847104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37ef63fceda47a8863cdab1b904c2be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.848780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ed475ccc-8630-4114-a4fb-70069886be7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37ef63fceda47a8863cdab1b904c2be'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.850084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e7cec-c73e-470b-b095-ab5ba998bb17 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:34.857942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:34.859513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:34.859971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:34.860437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] PATH_INFO: `/v3/users/008ac89bb1bc4eb787a710818df779c5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:34.862232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=008ac89bb1bc4eb787a710818df779c5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:34.865247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:34.884091 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1085/2166] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:34 2022] DELETE /identity/v3/users/a37ef63fceda47a8863cdab1b904c2be => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:34.887780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11909487-2307-4c8c-bc21-ceb56b3187dd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:34.888851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11909487-2307-4c8c-bc21-ceb56b3187dd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:34.889120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11909487-2307-4c8c-bc21-ceb56b3187dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:34.940836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-11909487-2307-4c8c-bc21-ceb56b3187dd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:34.942000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11909487-2307-4c8c-bc21-ceb56b3187dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:34.964373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:34.971912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008ac89bb1bc4eb787a710818df779c5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.974610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008ac89bb1bc4eb787a710818df779c5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:34.983935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:34.997249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008ac89bb1bc4eb787a710818df779c5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:35.021048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008ac89bb1bc4eb787a710818df779c5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:35.023299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2cd6b2ce-b3ef-4ae1-919e-471421714f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008ac89bb1bc4eb787a710818df779c5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:35.084183 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1082/2167] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:34 2022] DELETE /identity/v3/users/008ac89bb1bc4eb787a710818df779c5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:35.085905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abf720-af2f-4349-9aff-afd802b7f752 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.086589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abf720-af2f-4349-9aff-afd802b7f752 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.086872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abf720-af2f-4349-9aff-afd802b7f752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.114937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b7abf720-af2f-4349-9aff-afd802b7f752 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:35.120442 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1086/2168] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:35.121052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b7abf720-af2f-4349-9aff-afd802b7f752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:35.121158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.121158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.121158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.165676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:35.167967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:35.294127 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1083/2169] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:35.298436 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1087/2170] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:35.334099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.334403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.334679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.347891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:35.356988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:35.404012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:35.405505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:35.442224 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1084/2171] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:35.443318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:35.446782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:35.563977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944214d-3851-4ebc-9a12-67d58d60ba97 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:35.576814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:35.577617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.578108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.578564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.603767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:35.605481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:35.633730 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1088/2172] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:35.649958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:35.661594 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db978dd1-9e84-4efa-9ec4-66873d0ac033 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:35.673980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cec9dd-f6c4-4201-805a-ad5643310c2a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:35.685794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f960ccd-2ac9-40d5-af0f-92c5ddb66d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:35.687521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:35.688651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.689198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.689659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.689807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f960ccd-2ac9-40d5-af0f-92c5ddb66d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.690372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f960ccd-2ac9-40d5-af0f-92c5ddb66d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.692048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f960ccd-2ac9-40d5-af0f-92c5ddb66d28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.693417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f960ccd-2ac9-40d5-af0f-92c5ddb66d28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:35.695080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f960ccd-2ac9-40d5-af0f-92c5ddb66d28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:35.699585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f960ccd-2ac9-40d5-af0f-92c5ddb66d28 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:35.720925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:35.723351 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:35.751105 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1089/2173] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:35.751105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.751105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.751105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.752263 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1085/2174] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:35.767445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:35.817103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a0c4d-b70c-46c5-816f-b272ac69e146 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:35.840058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:35.846811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:35.847330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:35.847783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:35.855564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:35.870733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:35.876155 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] MFA Rules not processed for user `9d73ba36238f449dbe08d8888920510f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:35.880869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:35.988512 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1086/2175] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:35 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:35.990059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:36.072471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:36.128898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:36.142074 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1090/2176] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:36.180847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:36.180847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:36.235182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642346-8f70-4777-8be6-a19dacdc7c65 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:36.262822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:36.263597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:36.263883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:36.264155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:36.264874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:36.266919 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:36.366719 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1087/2177] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:35 2022] POST /identity/v3/users => generated 292 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:36.374355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:36.405277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a416b9e-dae1-4f64-b00b-3b19e9335673 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:36.450291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:36.459229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:36.459836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:36.460791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:36.466311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:36.470272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:36.518145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b4cd9-e5e3-468b-9582-c669b25d757f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:36.518858 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1088/2178] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:36.525162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:36.574520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bd6d9-e517-4869-a386-5bd54d4cf6d0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:36.599023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:36.600061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:36.600663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:36.601623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:36.602821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:36.612680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:36.683674 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1089/2179] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:02:36.684712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:36.730470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:36.731234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:36.731401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:36.732290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:36.793184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18abe7-3c04-4fd9-adc9-8335f24fb6ee admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:36.810803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:36.811936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:36.812453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:36.813049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] PATH_INFO: `/v3/projects/f53d77003a86430b93638b729930f788/users/2413fa59e75741a68f391e55a8a7dc2e/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:36.865983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53d77003a86430b93638b729930f788, user_id=2413fa59e75741a68f391e55a8a7dc2e, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:36.869550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:36.897228 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1090/2180] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:36 2022] PUT /identity/v3/projects/f53d77003a86430b93638b729930f788/users/2413fa59e75741a68f391e55a8a7dc2e/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:36.897228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:36.903622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:36.925002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:36.929082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:36.950777 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1091/2181] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:36.952139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:36.955289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:36.986946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4e29d-8641-412b-bdca-554c269b3a90 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:37.015291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:37.080156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13a6bb-8f2b-422d-b71d-31a06e56e825 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:37.080156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:37.080156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:37.080156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:37.080953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] PATH_INFO: `/v3/users/1274140873d84281800293848f480c0b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:37.080953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1274140873d84281800293848f480c0b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:37.081265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:37.130116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:37.131059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:37.131468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:37.132532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] PATH_INFO: `/v3/projects/418681cc67234a4aa44f3f4d4f765fc8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:37.134162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=418681cc67234a4aa44f3f4d4f765fc8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:37.137093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:37.168404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1274140873d84281800293848f480c0b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.169980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1274140873d84281800293848f480c0b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.185966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1274140873d84281800293848f480c0b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.212010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1274140873d84281800293848f480c0b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.213565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff975229-6f1d-4336-8dbf-06498f4e7a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1274140873d84281800293848f480c0b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.221205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '418681cc67234a4aa44f3f4d4f765fc8', 'request_id': 'req-1241496c-2e65-407c-b52d-392dda2990ab'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.233569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '418681cc67234a4aa44f3f4d4f765fc8', 'request_id': 'req-1241496c-2e65-407c-b52d-392dda2990ab'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.236194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:37.246585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '418681cc67234a4aa44f3f4d4f765fc8', 'request_id': 'req-1241496c-2e65-407c-b52d-392dda2990ab'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.252172 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1091/2182] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:36 2022] DELETE /identity/v3/users/1274140873d84281800293848f480c0b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:37.259757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0e581-41f1-4613-b4b9-41c73b19c8b3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:37.260184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0e581-41f1-4613-b4b9-41c73b19c8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:37.260704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0e581-41f1-4613-b4b9-41c73b19c8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:37.270667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] The token cache is being invalidate because project 418681cc67234a4aa44f3f4d4f765fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:37.271175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.272248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.279224 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1092/2183] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:36 2022] DELETE /identity/v3/projects/418681cc67234a4aa44f3f4d4f765fc8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:37.279224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:37.281245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:37.307136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-12d0e581-41f1-4613-b4b9-41c73b19c8b3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:37.309882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-12d0e581-41f1-4613-b4b9-41c73b19c8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:37.420866 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1092/2184] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:37.423243 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:37.423730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:37.424149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:37.459619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241496c-2e65-407c-b52d-392dda2990ab admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:37.460131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:37.461886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:37.468745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:37.469867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:37.471690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:37.472172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] PATH_INFO: `/v3/projects/33642e8ad0c3498ab62b99d210fdf560` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:37.474565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33642e8ad0c3498ab62b99d210fdf560)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:37.478009 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1093/2185] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:37.478435 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:37.479666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:37.482046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:37.501035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e61d1f-8141-4ac2-80e7-c8ac2231fbfb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:37.511030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:37.512572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:37.513155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:37.513638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] PATH_INFO: `/v3/users/e98fb2afe1844db5b528d03eea0fef50` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:37.516364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98fb2afe1844db5b528d03eea0fef50)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:37.521645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:37.568717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:37.573311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33642e8ad0c3498ab62b99d210fdf560', 'request_id': 'req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.589925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33642e8ad0c3498ab62b99d210fdf560', 'request_id': 'req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.614521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33642e8ad0c3498ab62b99d210fdf560', 'request_id': 'req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.650060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] The token cache is being invalidate because project 33642e8ad0c3498ab62b99d210fdf560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:37.650690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.651981 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.660635 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1093/2186] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:37 2022] DELETE /identity/v3/projects/33642e8ad0c3498ab62b99d210fdf560 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:37.662127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:37.669488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:37.673731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98fb2afe1844db5b528d03eea0fef50'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.675766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98fb2afe1844db5b528d03eea0fef50'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.694065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98fb2afe1844db5b528d03eea0fef50'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.722269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98fb2afe1844db5b528d03eea0fef50'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.723639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98fb2afe1844db5b528d03eea0fef50'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:37.749476 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1094/2187] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:37 2022] DELETE /identity/v3/users/e98fb2afe1844db5b528d03eea0fef50 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:37.749476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:37.765461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:37.809764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bdce3-114d-41fb-b7b9-d217e07a0da2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:37.956252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62494ae-4963-4920-8f08-fa1270265610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:37.956934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62494ae-4963-4920-8f08-fa1270265610 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:37.957218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62494ae-4963-4920-8f08-fa1270265610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:37.957500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62494ae-4963-4920-8f08-fa1270265610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:37.968181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e62494ae-4963-4920-8f08-fa1270265610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:37.983464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb271973-25dc-43f0-ac1b-8d0b798a438f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:37.998217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:37.999683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.001450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.002126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.008346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:38.074340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:38.084558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62494ae-4963-4920-8f08-fa1270265610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:38.087401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62494ae-4963-4920-8f08-fa1270265610 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:38.102808 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1094/2188] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:38.104001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf465a-e751-4493-8897-285a737a0f64 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.104555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf465a-e751-4493-8897-285a737a0f64 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.104845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf465a-e751-4493-8897-285a737a0f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.138626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-85bf465a-e751-4493-8897-285a737a0f64 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:38.142160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85bf465a-e751-4493-8897-285a737a0f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:38.171955 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1095/2189] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:38.188517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d340bb-e88c-4246-98ff-9a4082502c54 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.189202 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d340bb-e88c-4246-98ff-9a4082502c54 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.189694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d340bb-e88c-4246-98ff-9a4082502c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.259067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-88d340bb-e88c-4246-98ff-9a4082502c54 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:38.269578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-88d340bb-e88c-4246-98ff-9a4082502c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:38.274281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:38.277761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:38.300884 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1095/2190] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:38.324969 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1096/2191] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:38.331981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:38.356558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dc3e4-f6f0-48f5-8bd7-9704cc3040e2 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:38.370223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:38.371183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.371495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.372050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.373423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:38.373850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:38.375652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:38.395617 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1096/2192] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:02:38.405580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d340bb-e88c-4246-98ff-9a4082502c54 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:38.429100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d340bb-e88c-4246-98ff-9a4082502c54 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:38.454996 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:38.456086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.456613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.457113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.458323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:38.473066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:38.575687 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1097/2193] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:38 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:38.576435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:38.592001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0dc978-1c34-4ce0-a8c1-b97d70e1ac2d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:38.634387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:38.638056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:38.659728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:38.705929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:38.707327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.708164 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.708874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.710884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:38.715283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:38.770290 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1097/2194] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:38 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:38.777119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:38.798402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2ea41-5862-48a3-9e16-0ed3e41e156a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:38.806175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:38.807215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.807750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.808190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] PATH_INFO: `/v3/projects/738b4076128a4b4c82061d193750e5b3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.810091 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738b4076128a4b4c82061d193750e5b3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:38.813507 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:38.819937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e89895-46ac-4755-8468-399ac7769dab admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:38.832111 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:38.835776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.836726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.837746 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.839298 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:38.842221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:38.869710 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1098/2195] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:38.876984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:38.885453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738b4076128a4b4c82061d193750e5b3', 'request_id': 'req-19cb5d9d-f06c-4a96-93a5-7121c95f2233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:38.893729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738b4076128a4b4c82061d193750e5b3', 'request_id': 'req-19cb5d9d-f06c-4a96-93a5-7121c95f2233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:38.903984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738b4076128a4b4c82061d193750e5b3', 'request_id': 'req-19cb5d9d-f06c-4a96-93a5-7121c95f2233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:38.904765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad1bd-30db-4983-8856-b48d5bf0a127 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:38.936075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] The token cache is being invalidate because project 738b4076128a4b4c82061d193750e5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:38.937259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:38.938300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-19cb5d9d-f06c-4a96-93a5-7121c95f2233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:38.941011 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1098/2196] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:38 2022] DELETE /identity/v3/projects/738b4076128a4b4c82061d193750e5b3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:38.948589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c0240-0460-42e3-ab95-44e74426a695 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:38.949860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c0240-0460-42e3-ab95-44e74426a695 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:38.950380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c0240-0460-42e3-ab95-44e74426a695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:38.988663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8f5c0240-0460-42e3-ab95-44e74426a695 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:38.990624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8f5c0240-0460-42e3-ab95-44e74426a695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:39.086560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.087897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.088894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.089814 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1099/2197] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:39.092935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.097747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.102371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d1650-7ce0-41f6-b24b-a478b56d90d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.103530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d1650-7ce0-41f6-b24b-a478b56d90d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.103530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d1650-7ce0-41f6-b24b-a478b56d90d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.105570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.128457 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1099/2198] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:39.131547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.135803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:39.145411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3a0d1650-7ce0-41f6-b24b-a478b56d90d4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:39.147157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a0d1650-7ce0-41f6-b24b-a478b56d90d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:39.159105 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1100/2199] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:39.166615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d1650-7ce0-41f6-b24b-a478b56d90d4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.175654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817e7ae-a592-479a-ad88-0107ba269828 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:39.183561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d1650-7ce0-41f6-b24b-a478b56d90d4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:39.190544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c57ce-346b-416d-8224-522e645b12a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.191481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.192107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c57ce-346b-416d-8224-522e645b12a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.192657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c57ce-346b-416d-8224-522e645b12a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.192968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.193426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c57ce-346b-416d-8224-522e645b12a2 admin admin] PATH_INFO: `/v3/projects/a49eb5e19c0e44b3b9273f5126c0915f/users/1bc4010eab744cf78053adb904f85dc5/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.193628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.194053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.196121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:02:39.196679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.200829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.219451 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1101/2200] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:02:39.226714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:39.227363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.234071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c57ce-346b-416d-8224-522e645b12a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49eb5e19c0e44b3b9273f5126c0915f, user_id=1bc4010eab744cf78053adb904f85dc5, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.238909 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c57ce-346b-416d-8224-522e645b12a2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.264615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6508b8-2c49-4024-9e1d-168c5287c8ae admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:39.272039 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1100/2201] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:39 2022] PUT /identity/v3/projects/a49eb5e19c0e44b3b9273f5126c0915f/users/1bc4010eab744cf78053adb904f85dc5/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:02:39.282379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.282379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.283214 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.324450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:39.327425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:39.380637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:39.483672 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1101/2202] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:39.509310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.511787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:39.576254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.579824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.582311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.582613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.583313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.590133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.671971 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1102/2203] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:39 2022] POST /identity/v3/projects => generated 373 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:39.674334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.700478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7311cd-4fbd-4f03-8191-efc4ca2f5b78 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:39.792143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db198a8f-2cd6-4d14-9b09-e2e726eee01a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:39.821041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.822096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.822590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.823259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.824553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.828730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.874543 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1103/2204] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:39 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:39.874779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.874779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.874779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.874779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.876489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.905464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.908740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.912274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0417a7-e711-4a76-b75b-a2f982aca1f0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:39.925812 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1102/2205] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:39.927543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.930385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:39.931338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.931995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.932298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.932618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.933376 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.935222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.955390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c82d8-a078-4245-97e4-d4fb8d6c2a0c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:39.965615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:39.967101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:39.970355 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1104/2206] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:39.970475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:39.970475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] PATH_INFO: `/v3/projects/bb6109a0a7754f4f9771b1bd403e248d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:39.971114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6109a0a7754f4f9771b1bd403e248d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:39.971765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:39.977416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:39.997822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb23a21-dd55-4b09-86ee-adbe8df3113d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:40.018785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:40.019789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:40.020188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:40.020509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:40.021277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:40.023678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:40.057410 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1105/2207] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:40.058024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:40.103177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7de8ef-64e6-4637-a769-a8aa30fd7a03 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:40.158105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:40.171826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6109a0a7754f4f9771b1bd403e248d', 'request_id': 'req-ab430e5a-1bbf-4c7f-8408-a2d905fce743'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:40.198304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6109a0a7754f4f9771b1bd403e248d', 'request_id': 'req-ab430e5a-1bbf-4c7f-8408-a2d905fce743'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:40.227799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6109a0a7754f4f9771b1bd403e248d', 'request_id': 'req-ab430e5a-1bbf-4c7f-8408-a2d905fce743'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:40.277841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] The token cache is being invalidate because project bb6109a0a7754f4f9771b1bd403e248d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:40.279843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:40.282856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab430e5a-1bbf-4c7f-8408-a2d905fce743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:40.302037 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1103/2208] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:39 2022] DELETE /identity/v3/projects/bb6109a0a7754f4f9771b1bd403e248d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:40.389478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:40.392917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:40.393275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:40.396494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] PATH_INFO: `/v3/projects/5eaa9877bdda4c17b42121afda7e8b46/users/5292802f9cb14892aad2a00d3eb7ac54/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:40.448655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:40.468675 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaa9877bdda4c17b42121afda7e8b46, user_id=5292802f9cb14892aad2a00d3eb7ac54, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:40.475122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:40.527726 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1106/2209] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:40 2022] PUT /identity/v3/projects/5eaa9877bdda4c17b42121afda7e8b46/users/5292802f9cb14892aad2a00d3eb7ac54/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:40.534025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:40.534883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:40.535526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:40.620076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] MFA Rules not processed for user `5292802f9cb14892aad2a00d3eb7ac54`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:40.623601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:40.791507 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1104/2210] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:40.800701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:40.802700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:40.925265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31399b5-51b9-41c4-8771-1434f2947776 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:40.936566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4667b-f36e-4668-bab3-b006df200c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:40.937872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4667b-f36e-4668-bab3-b006df200c14 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:40.938473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4667b-f36e-4668-bab3-b006df200c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:40.939306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4667b-f36e-4668-bab3-b006df200c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:40.958158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4667b-f36e-4668-bab3-b006df200c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:40.963685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4667b-f36e-4668-bab3-b006df200c14 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:41.009503 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1107/2211] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:41.036548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:41.040262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:41.290478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409578d-3513-4d35-b0b5-3854973036a3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:41.303615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:41.306163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:41.306713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:41.307905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] PATH_INFO: `/v3/users/5292802f9cb14892aad2a00d3eb7ac54` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:41.310118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5292802f9cb14892aad2a00d3eb7ac54)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:41.315436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:41.515713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5292802f9cb14892aad2a00d3eb7ac54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:41.521682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5292802f9cb14892aad2a00d3eb7ac54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:41.570229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5292802f9cb14892aad2a00d3eb7ac54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:41.575125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00b95-e432-4980-8de9-639a8dfb3b86 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:41.575619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00b95-e432-4980-8de9-639a8dfb3b86 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:41.576089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00b95-e432-4980-8de9-639a8dfb3b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:41.618022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:41.656060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5292802f9cb14892aad2a00d3eb7ac54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:41.657540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-557ffd08-5d4e-4d96-bfe6-922b46c17fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5292802f9cb14892aad2a00d3eb7ac54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:41.699327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fdd00b95-e432-4980-8de9-639a8dfb3b86 None None] MFA Rules not processed for user `2413fa59e75741a68f391e55a8a7dc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:41.702621 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1105/2212] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:02:41 2022] DELETE /identity/v3/users/5292802f9cb14892aad2a00d3eb7ac54 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:41.707839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fdd00b95-e432-4980-8de9-639a8dfb3b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:41.711635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:41.712560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:41.714901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:41.768946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:41.770514 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:41.959835 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1106/2213] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:41.984529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:41.986314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:42.032178 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1108/2214] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:42.050128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:42.051624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:42.053430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:42.120736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] MFA Rules not processed for user `9d73ba36238f449dbe08d8888920510f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:42.123400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:42.132281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3baa2-e22f-4dcd-9e7f-1b504b42eacf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:42.138860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:42.139957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:42.140563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:42.141447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:42.157197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:42.159894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:42.176032 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1107/2215] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:02:42.177220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:42.180071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:42.325555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d9478-ba0a-413a-b6fd-b93f795c0371 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:42.332021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807350e-6364-418d-9592-c4ffb359255e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:42.333446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807350e-6364-418d-9592-c4ffb359255e service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:42.333743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807350e-6364-418d-9592-c4ffb359255e service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:42.334031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807350e-6364-418d-9592-c4ffb359255e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:42.356848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807350e-6364-418d-9592-c4ffb359255e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:42.358609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807350e-6364-418d-9592-c4ffb359255e service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:42.368277 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1109/2216] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:42.384457 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1108/2217] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:02:44.033566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:44.035290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:44.065895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53882699-b798-4666-91b4-97b2f93b0f12 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:44.082764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:44.083568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:44.083948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:44.084485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] PATH_INFO: `/v3/projects/5eaa9877bdda4c17b42121afda7e8b46` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:44.087930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eaa9877bdda4c17b42121afda7e8b46)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:44.093854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:44.192069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:44.262566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eaa9877bdda4c17b42121afda7e8b46', 'request_id': 'req-3f076492-59c0-48bb-8567-627abc01a345'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:44.288462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eaa9877bdda4c17b42121afda7e8b46', 'request_id': 'req-3f076492-59c0-48bb-8567-627abc01a345'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:44.323918 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eaa9877bdda4c17b42121afda7e8b46', 'request_id': 'req-3f076492-59c0-48bb-8567-627abc01a345'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:44.356114 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] The token cache is being invalidate because project 5eaa9877bdda4c17b42121afda7e8b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:02:44.356492 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:44.357943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3f076492-59c0-48bb-8567-627abc01a345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:02:44.360995 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1110/2218] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:02:44 2022] DELETE /identity/v3/projects/5eaa9877bdda4c17b42121afda7e8b46 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:44.568939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:44.571050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:44.571767 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:44.645836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] MFA Rules not processed for user `2413fa59e75741a68f391e55a8a7dc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:44.652546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:44.724331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:44.814685 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1109/2219] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:02:51.077815 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edd2b7-dc00-4a45-8661-1e548fe948fa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:51.078422 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edd2b7-dc00-4a45-8661-1e548fe948fa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:51.079182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edd2b7-dc00-4a45-8661-1e548fe948fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:51.136949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-71edd2b7-dc00-4a45-8661-1e548fe948fa None None] MFA Rules not processed for user `1bc4010eab744cf78053adb904f85dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:02:51.137917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-71edd2b7-dc00-4a45-8661-1e548fe948fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:51.237187 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1111/2220] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:02:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:02:51.251629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:51.253432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:02:51.390263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848f5de-a34b-430c-970f-4b8527945cbe None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:51.407401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:51.408992 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:51.410295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:51.410670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:51.411453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:51.414609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:51.466144 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1110/2221] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:02:51 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:02:51.470219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edd2b7-dc00-4a45-8661-1e548fe948fa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:51.492905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edd2b7-dc00-4a45-8661-1e548fe948fa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:51.600478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:51.603368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:51.605982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:51.607145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:51.609548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:51.614459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:51.672878 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1112/2222] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:02:51 2022] POST /identity/v3/users => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:02:51.673616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:51.697197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68394e7e-d0e2-4176-9a68-0688119ef50b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:51.708366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:51.709882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:51.710930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:51.711477 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:51.712331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:51.715945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:51.733931 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1111/2223] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:02:51.739121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:51.752445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301559f4-3289-4f56-bb3c-94f62f2531dd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:51.757922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:51.758637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:51.758910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:51.759182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] PATH_INFO: `/v3/projects/e694c5b3e92841958444552023c7d701/users/3c5febc3151e4d149b49578a39ceeba8/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:51.779293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e694c5b3e92841958444552023c7d701, user_id=3c5febc3151e4d149b49578a39ceeba8, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:51.782160 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:51.807268 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1113/2224] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:51 2022] PUT /identity/v3/projects/e694c5b3e92841958444552023c7d701/users/3c5febc3151e4d149b49578a39ceeba8/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:02:51.816854 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:51.835834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0322b-4e78-4a7e-b234-a7f99e4c8453 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:51.956286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85b943-3362-46c4-8eb3-3c0f538c2682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:51.957420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85b943-3362-46c4-8eb3-3c0f538c2682 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:51.957913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85b943-3362-46c4-8eb3-3c0f538c2682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:51.958397 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85b943-3362-46c4-8eb3-3c0f538c2682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:51.960042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85b943-3362-46c4-8eb3-3c0f538c2682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:51.963109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c85b943-3362-46c4-8eb3-3c0f538c2682 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:51.988491 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1112/2225] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:02:51.993914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:02:52.021374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d00bf-4348-4d18-a542-cf3a9c1e7754 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:02:52.031540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:02:52.032749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:02:52.033518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:02:52.034006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] PATH_INFO: `/v3/projects/e694c5b3e92841958444552023c7d701/users/3c5febc3151e4d149b49578a39ceeba8/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:02:52.036907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e694c5b3e92841958444552023c7d701, user_id=3c5febc3151e4d149b49578a39ceeba8, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:02:52.044147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:02:52.048659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:52.119556 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1114/2226] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:02:51 2022] PUT /identity/v3/projects/e694c5b3e92841958444552023c7d701/users/3c5febc3151e4d149b49578a39ceeba8/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:03:00.432497 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026f24b-adfc-4497-8adb-4b0f13a150d4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:00.433052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026f24b-adfc-4497-8adb-4b0f13a150d4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:00.433439 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026f24b-adfc-4497-8adb-4b0f13a150d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:00.485630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d026f24b-adfc-4497-8adb-4b0f13a150d4 None None] MFA Rules not processed for user `3c5febc3151e4d149b49578a39ceeba8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:00.487169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d026f24b-adfc-4497-8adb-4b0f13a150d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:00.543815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:00.625129 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1113/2227] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:03:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:03:00.637801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:03:00.639804 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:00.744310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd3b66-61d5-46f7-857b-db273106eb11 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:03:00.751513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:03:00.752602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:00.753108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:00.753611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:00.910099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:03:00.911794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:03:00.926817 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1115/2228] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:03:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:03:01.090616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f62ed6-7613-4e4c-b870-b57a0f2e5d76 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:01.090959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f62ed6-7613-4e4c-b870-b57a0f2e5d76 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:01.091307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f62ed6-7613-4e4c-b870-b57a0f2e5d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:01.116106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-34f62ed6-7613-4e4c-b870-b57a0f2e5d76 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:01.120096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34f62ed6-7613-4e4c-b870-b57a0f2e5d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:01.215015 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1114/2229] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:03:01.273880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:03:01.276643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:01.421860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d225cf-f886-43a4-817b-958f2c8b812f service glance] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:03:01.437619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32012930-a586-432c-a3aa-bc94cdec9c44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:03:01.439304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012930-a586-432c-a3aa-bc94cdec9c44 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:01.439665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012930-a586-432c-a3aa-bc94cdec9c44 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:01.440162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012930-a586-432c-a3aa-bc94cdec9c44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:01.452220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32012930-a586-432c-a3aa-bc94cdec9c44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:03:01.454452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32012930-a586-432c-a3aa-bc94cdec9c44 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:03:01.498097 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1116/2230] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:03:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:03:11.207468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=39 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:11.466865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=1,keystone:SELECT=82 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:17.018615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e8ee7-009a-4a48-92d6-abecab219087 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:17.019430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e8ee7-009a-4a48-92d6-abecab219087 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:17.019430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e8ee7-009a-4a48-92d6-abecab219087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:17.065096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4b4e8ee7-009a-4a48-92d6-abecab219087 None None] MFA Rules not processed for user `1bc4010eab744cf78053adb904f85dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:17.067544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4b4e8ee7-009a-4a48-92d6-abecab219087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:17.084427 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1115/2231] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:03:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:03:17.495567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff91884-39c9-4d52-80db-3a47d05343b6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:17.496354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff91884-39c9-4d52-80db-3a47d05343b6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:17.496354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff91884-39c9-4d52-80db-3a47d05343b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:17.518085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3ff91884-39c9-4d52-80db-3a47d05343b6 None None] MFA Rules not processed for user `ca20e105b76149dd99e4aa64c62eb483`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:17.519986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ff91884-39c9-4d52-80db-3a47d05343b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:17.601606 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1117/2232] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:03:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:03:18.899835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7782-ff2d-4cdb-80f2-3ef57f1dff36 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:18.900592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7782-ff2d-4cdb-80f2-3ef57f1dff36 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:18.900592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7782-ff2d-4cdb-80f2-3ef57f1dff36 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:18.902620 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1116/2233] 10.222.0.26 () {54 vars in 961 bytes} [Sun Sep 25 01:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:03:18.909951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebb027-c6e4-4801-905a-7606a8fb41b6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:18.910831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebb027-c6e4-4801-905a-7606a8fb41b6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:18.910831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebb027-c6e4-4801-905a-7606a8fb41b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:18.944101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-aeebb027-c6e4-4801-905a-7606a8fb41b6 None None] MFA Rules not processed for user `d45df63e9f2449be8d13f58100775eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:18.946703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aeebb027-c6e4-4801-905a-7606a8fb41b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:19.068077 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1118/2234] 10.222.0.26 () {58 vars in 1058 bytes} [Sun Sep 25 01:03:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:03:19.082442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d7782-ff2d-4cdb-80f2-3ef57f1dff36 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:03:19.103245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d7782-ff2d-4cdb-80f2-3ef57f1dff36 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:03:19.112251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:03:19.113586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:19.114263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:19.114794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:19.118186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:19.155074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:03:19.160974 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:03:19.178562 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1117/2235] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:03:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:03:20.924494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f42f4c-355b-40bb-b227-df37a3a42843 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:20.925474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f42f4c-355b-40bb-b227-df37a3a42843 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:20.925474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f42f4c-355b-40bb-b227-df37a3a42843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:20.972572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-16f42f4c-355b-40bb-b227-df37a3a42843 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:20.974687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-16f42f4c-355b-40bb-b227-df37a3a42843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:20.992910 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1119/2236] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:03:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:03:21.007861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:03:21.045469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d41931-581e-495f-b268-dcfc6f48e9e2 service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:03:21.056941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:03:21.058749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:21.059675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:21.060944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:21.081131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:03:21.083567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:03:21.095260 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1118/2237] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:03:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 25 01:03:30.961957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=57 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:31.090401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=15 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:44.682216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3b8fa-4ea2-4d09-9c9e-6300be24b00a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:44.682902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3b8fa-4ea2-4d09-9c9e-6300be24b00a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:44.683113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3b8fa-4ea2-4d09-9c9e-6300be24b00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:44.708482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8fa3b8fa-4ea2-4d09-9c9e-6300be24b00a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:44.709630 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8fa3b8fa-4ea2-4d09-9c9e-6300be24b00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:44.806201 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1120/2238] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:03:44.834273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:03:44.838547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:44.934590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdbcab-2705-412f-b517-fa983a0646c7 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:03:44.940461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:03:44.941113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:44.941395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:44.941673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:44.951227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:03:44.952912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:03:44.966098 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1119/2239] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:03:54.105769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5f7e-4b68-44a9-8e54-69a75e6ba0c3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:54.106595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5f7e-4b68-44a9-8e54-69a75e6ba0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:54.106595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5f7e-4b68-44a9-8e54-69a75e6ba0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:54.134987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a88d5f7e-4b68-44a9-8e54-69a75e6ba0c3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:54.137667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a88d5f7e-4b68-44a9-8e54-69a75e6ba0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:54.155094 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1121/2240] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:03:54.177916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:03:54.179891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:54.197616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24f48-8aea-4242-b47f-744f0dedcd1f service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:03:54.203949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:03:54.204673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:54.204993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:54.205312 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:54.217232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:03:54.219031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:03:54.232716 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1120/2241] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:03:59.681867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde8c54-67c4-436e-9bb7-ac9976bf9f59 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:59.682847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde8c54-67c4-436e-9bb7-ac9976bf9f59 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:59.682847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde8c54-67c4-436e-9bb7-ac9976bf9f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:59.710476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1dde8c54-67c4-436e-9bb7-ac9976bf9f59 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:03:59.711868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1dde8c54-67c4-436e-9bb7-ac9976bf9f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:03:59.723023 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1122/2242] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:03:59.955519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:03:59.970751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a3fd4-d23b-4ba8-8f17-de52ac261da9 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:03:59.978744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:03:59.979520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:03:59.980038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:03:59.980445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:03:59.992610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:03:59.994288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:00.005595 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1121/2243] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:04:03.790169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:03.790730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:03.791091 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:03.816400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:03.817781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:03.826587 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1123/2244] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:04:03.874021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:03.887457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10cf0-0536-41fc-8d9d-e34b51148703 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:03.894051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bd332-d029-4554-9da1-de35ed7cd703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:03.894670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bd332-d029-4554-9da1-de35ed7cd703 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:03.895156 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bd332-d029-4554-9da1-de35ed7cd703 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:03.895425 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bd332-d029-4554-9da1-de35ed7cd703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:03.911710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bd332-d029-4554-9da1-de35ed7cd703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:03.915684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bd332-d029-4554-9da1-de35ed7cd703 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:03.931490 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1122/2245] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:04:09.125346 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:09.127284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:09.176491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a342-3588-4dba-afd2-79bd6fc12131 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:09.177164 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a342-3588-4dba-afd2-79bd6fc12131 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:09.177473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723a342-3588-4dba-afd2-79bd6fc12131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:09.205923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d723a342-3588-4dba-afd2-79bd6fc12131 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:09.207490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d723a342-3588-4dba-afd2-79bd6fc12131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:09.208567 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bc83e-11f5-48ff-91a0-504883a95c39 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:09.214228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d85b1a-c166-4dce-bce7-f4d007c49a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:09.214228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d85b1a-c166-4dce-bce7-f4d007c49a19 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:09.214228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d85b1a-c166-4dce-bce7-f4d007c49a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:09.214910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d85b1a-c166-4dce-bce7-f4d007c49a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:09.219713 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1123/2246] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:04:09.227156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d85b1a-c166-4dce-bce7-f4d007c49a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:09.229128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d85b1a-c166-4dce-bce7-f4d007c49a19 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:09.240751 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1124/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:04:09.251074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723a342-3588-4dba-afd2-79bd6fc12131 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:09.274757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723a342-3588-4dba-afd2-79bd6fc12131 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:09.285712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:09.287047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:09.287621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:09.288712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:09.309392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:09.312328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:09.331239 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1124/2248] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:04:13.485664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8fa1b-bf93-4ac9-aaed-348c8b009969 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:13.486535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8fa1b-bf93-4ac9-aaed-348c8b009969 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:13.486904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8fa1b-bf93-4ac9-aaed-348c8b009969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:13.506334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=63 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:13.540293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-07d8fa1b-bf93-4ac9-aaed-348c8b009969 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:13.542611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-07d8fa1b-bf93-4ac9-aaed-348c8b009969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:13.582496 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1125/2249] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:04:13.627404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:13.634344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=62 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:13.642925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2037c24-3c1b-48dd-87cd-bdccd4343815 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:13.649898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:13.651476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:13.651755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:13.652244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:13.666051 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:13.669513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:13.690932 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1125/2250] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:04:23.527045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:23.680907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:30.061980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963520b2-2309-4002-8560-d2bbe70b888d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:30.062701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963520b2-2309-4002-8560-d2bbe70b888d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:30.062701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963520b2-2309-4002-8560-d2bbe70b888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:30.086662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-963520b2-2309-4002-8560-d2bbe70b888d None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:30.088167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-963520b2-2309-4002-8560-d2bbe70b888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:30.104426 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1126/2251] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:04:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:04:30.162376 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:30.176283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ba748-acb2-4a5e-813a-953cb377b21b service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:30.182466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dbfaf-c571-45ce-aa65-d2bd91772ccd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:30.183100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dbfaf-c571-45ce-aa65-d2bd91772ccd service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:30.183384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dbfaf-c571-45ce-aa65-d2bd91772ccd service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:30.183659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dbfaf-c571-45ce-aa65-d2bd91772ccd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:30.194767 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dbfaf-c571-45ce-aa65-d2bd91772ccd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:30.196666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1dbfaf-c571-45ce-aa65-d2bd91772ccd service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:30.211189 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1126/2252] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:04:35.298092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fbdec-c1c1-49dc-abdf-b5c395ba29ff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:35.298935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fbdec-c1c1-49dc-abdf-b5c395ba29ff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:35.298935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fbdec-c1c1-49dc-abdf-b5c395ba29ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:35.334733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f60fbdec-c1c1-49dc-abdf-b5c395ba29ff None None] MFA Rules not processed for user `2413fa59e75741a68f391e55a8a7dc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:35.337171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f60fbdec-c1c1-49dc-abdf-b5c395ba29ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:35.436918 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1127/2253] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:04:40.204520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:45.423232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=28 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:52.003012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5737b56-b38c-4f7e-b453-55054e4e877f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.003952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5737b56-b38c-4f7e-b453-55054e4e877f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.003952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5737b56-b38c-4f7e-b453-55054e4e877f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.006063 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1127/2254] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:04:52.014181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fc140-16d0-49e1-b94b-576f5997089e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.015026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fc140-16d0-49e1-b94b-576f5997089e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.015204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fc140-16d0-49e1-b94b-576f5997089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.052735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6b4fc140-16d0-49e1-b94b-576f5997089e None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:52.055331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6b4fc140-16d0-49e1-b94b-576f5997089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:52.073461 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1128/2255] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:04:52.093818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5737b56-b38c-4f7e-b453-55054e4e877f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:52.111736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5737b56-b38c-4f7e-b453-55054e4e877f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:52.116064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65575d3-77b4-4868-b0ca-b4fb0ce653ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:52.117337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65575d3-77b4-4868-b0ca-b4fb0ce653ae service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.117588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65575d3-77b4-4868-b0ca-b4fb0ce653ae service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.117947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65575d3-77b4-4868-b0ca-b4fb0ce653ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.130404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65575d3-77b4-4868-b0ca-b4fb0ce653ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:52.132107 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65575d3-77b4-4868-b0ca-b4fb0ce653ae service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:52.146595 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1128/2256] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:04:52.253037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b41c1-c8f5-4faa-ba48-05ddbeda9d1e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.253438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b41c1-c8f5-4faa-ba48-05ddbeda9d1e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.254010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b41c1-c8f5-4faa-ba48-05ddbeda9d1e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.255451 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1129/2257] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 25 01:04:52.263432 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11078b99-85cf-4fc3-95b9-ec0cdf32da3a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.263804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11078b99-85cf-4fc3-95b9-ec0cdf32da3a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.264153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11078b99-85cf-4fc3-95b9-ec0cdf32da3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.290603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-11078b99-85cf-4fc3-95b9-ec0cdf32da3a None None] MFA Rules not processed for user `8427b2d4e8c040d0a1b00e25e3b58e30`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:52.291866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11078b99-85cf-4fc3-95b9-ec0cdf32da3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:52.299430 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1129/2258] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:04:52.313014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b41c1-c8f5-4faa-ba48-05ddbeda9d1e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:52.314854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-069b41c1-c8f5-4faa-ba48-05ddbeda9d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:52.332069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b41c1-c8f5-4faa-ba48-05ddbeda9d1e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:52.338755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ff6ed-243c-495f-9068-30f492b06d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:52.339878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ff6ed-243c-495f-9068-30f492b06d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.340364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ff6ed-243c-495f-9068-30f492b06d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.340881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ff6ed-243c-495f-9068-30f492b06d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.356013 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ff6ed-243c-495f-9068-30f492b06d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:52.358026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ff6ed-243c-495f-9068-30f492b06d2f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:52.372005 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1130/2259] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:04:52.489447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a148-9c2a-4ac3-9daf-76b545d470b2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.489886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a148-9c2a-4ac3-9daf-76b545d470b2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.490253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a148-9c2a-4ac3-9daf-76b545d470b2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.493248 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1130/2260] 10.222.0.26 () {54 vars in 921 bytes} [Sun Sep 25 01:04:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 25 01:04:52.506472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65c4eb-c8be-44ac-8501-8c7eec6d0871 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.507005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65c4eb-c8be-44ac-8501-8c7eec6d0871 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.507414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65c4eb-c8be-44ac-8501-8c7eec6d0871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.541872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fc65c4eb-c8be-44ac-8501-8c7eec6d0871 None None] MFA Rules not processed for user `9d8bf513dc544632a09a2d10988a888f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:52.543216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fc65c4eb-c8be-44ac-8501-8c7eec6d0871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:52.554891 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1131/2261] 10.222.0.26 () {58 vars in 1018 bytes} [Sun Sep 25 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:04:52.571959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a148-9c2a-4ac3-9daf-76b545d470b2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:52.586633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a148-9c2a-4ac3-9daf-76b545d470b2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:52.591314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:52.592302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:52.592863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:52.593137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:52.607102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:52.608631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:52.625498 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1131/2262] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:04:57.613469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ee3d0-f71f-4465-af8d-915f5ab14786 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:57.614314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ee3d0-f71f-4465-af8d-915f5ab14786 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:57.615032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ee3d0-f71f-4465-af8d-915f5ab14786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:57.654518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-318ee3d0-f71f-4465-af8d-915f5ab14786 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:04:57.656790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-318ee3d0-f71f-4465-af8d-915f5ab14786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:04:57.672614 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1132/2263] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:04:57.707227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:04:57.722858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aafa2-b072-4702-ba32-c8dfe218f736 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:04:57.732766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:04:57.733759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:04:57.734231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:04:57.734687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:04:57.751202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:04:57.753452 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:04:57.766274 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1132/2264] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:05:00.960082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143b4d2-0c85-499c-a6d0-4dc48f475894 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:00.963969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143b4d2-0c85-499c-a6d0-4dc48f475894 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:00.963969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143b4d2-0c85-499c-a6d0-4dc48f475894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:01.009500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8143b4d2-0c85-499c-a6d0-4dc48f475894 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:05:01.011681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8143b4d2-0c85-499c-a6d0-4dc48f475894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:01.026679 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1133/2265] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:05:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:05:01.048321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:01.070978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b96c4-584a-4e72-bfa5-41ab9f14d3da service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:01.080024 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:01.080847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:01.081223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:01.081608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:01.092506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:01.094486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:01.106481 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1133/2266] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:05:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:05:03.466219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca978e94-0532-4775-bb17-9082d9a23ee4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:03.469654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca978e94-0532-4775-bb17-9082d9a23ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:03.471091 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca978e94-0532-4775-bb17-9082d9a23ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:03.537488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ca978e94-0532-4775-bb17-9082d9a23ee4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:05:03.541954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca978e94-0532-4775-bb17-9082d9a23ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:03.562863 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1134/2267] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:05:03.594669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:03.624139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55328d0-378c-4ae4-a31c-5c293f9cd006 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:03.635567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556ce80-8f7d-4838-a346-ac02ee7909d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:03.638062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556ce80-8f7d-4838-a346-ac02ee7909d4 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:03.638817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556ce80-8f7d-4838-a346-ac02ee7909d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:03.639512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556ce80-8f7d-4838-a346-ac02ee7909d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:03.654204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556ce80-8f7d-4838-a346-ac02ee7909d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:03.659527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556ce80-8f7d-4838-a346-ac02ee7909d4 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:03.678851 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1134/2268] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:05:04.622505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318f63-dbf6-47a3-8d2a-51d8aa4c1e1f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:04.623561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318f63-dbf6-47a3-8d2a-51d8aa4c1e1f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:04.623561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318f63-dbf6-47a3-8d2a-51d8aa4c1e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:04.688887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-33318f63-dbf6-47a3-8d2a-51d8aa4c1e1f None None] MFA Rules not processed for user `9d73ba36238f449dbe08d8888920510f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:05:04.692667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-33318f63-dbf6-47a3-8d2a-51d8aa4c1e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:04.831645 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1135/2269] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:05:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:05:13.670075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=33 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:14.814022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=46 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:25.511624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede961-6b9a-4d2b-bce0-6c3f8d67efb1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:25.512483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede961-6b9a-4d2b-bce0-6c3f8d67efb1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:25.513029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede961-6b9a-4d2b-bce0-6c3f8d67efb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:25.559110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-29ede961-6b9a-4d2b-bce0-6c3f8d67efb1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:05:25.561517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29ede961-6b9a-4d2b-bce0-6c3f8d67efb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:25.578441 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1135/2270] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:05:25.617607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33318f63-dbf6-47a3-8d2a-51d8aa4c1e1f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:25.671354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33318f63-dbf6-47a3-8d2a-51d8aa4c1e1f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:25.690463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:25.695221 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:25.696634 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:25.700639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:25.724035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:25.729133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:25.758870 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1136/2271] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:05:29.158733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6bc1-466c-41fe-bbb3-ddf6b9ba2a89 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:29.162988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6bc1-466c-41fe-bbb3-ddf6b9ba2a89 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:29.163640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6bc1-466c-41fe-bbb3-ddf6b9ba2a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:29.216754 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-332f6bc1-466c-41fe-bbb3-ddf6b9ba2a89 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:05:29.219026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-332f6bc1-466c-41fe-bbb3-ddf6b9ba2a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:29.229469 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1136/2272] 10.222.0.26 () {58 vars in 967 bytes} [Sun Sep 25 01:05:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:05:29.251921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:29.272698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35de49-345d-4f29-98ed-d114f9832461 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:29.279024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:29.280037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:29.280552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:29.281092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:29.298227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:29.301417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:29.317816 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1137/2273] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:05:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:05:30.083740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6bc1-466c-41fe-bbb3-ddf6b9ba2a89 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:30.097768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6bc1-466c-41fe-bbb3-ddf6b9ba2a89 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:30.103173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfa3bf-267a-43f9-9743-1cfa6b3c5127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:30.103999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfa3bf-267a-43f9-9743-1cfa6b3c5127 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:30.104304 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfa3bf-267a-43f9-9743-1cfa6b3c5127 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:30.104611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfa3bf-267a-43f9-9743-1cfa6b3c5127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:30.107126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-67dfa3bf-267a-43f9-9743-1cfa6b3c5127 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:30.125565 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfa3bf-267a-43f9-9743-1cfa6b3c5127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:30.127412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfa3bf-267a-43f9-9743-1cfa6b3c5127 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:30.143196 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1137/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:05:30.504283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:30.517292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881ffa-3ca1-4223-8b94-cea2e3aea579 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:30.523756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:30.524571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:30.524920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:30.525217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:30.527237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:30.552436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:30.555382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:30.578397 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1138/2275] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:05:31.926994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1859-d75e-4dff-baea-e4a50e545f83 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:31.927665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1859-d75e-4dff-baea-e4a50e545f83 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:31.928138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1859-d75e-4dff-baea-e4a50e545f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:31.962740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-321a1859-d75e-4dff-baea-e4a50e545f83 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:05:31.965009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-321a1859-d75e-4dff-baea-e4a50e545f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:31.977873 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1138/2276] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:05:32.001426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:32.014707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161eab6c-a62d-4cac-a1e0-550e31f7d930 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:32.021140 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:32.021789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:32.022250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:32.023187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:32.033472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:32.035468 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:32.050207 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1139/2277] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:05:38.386875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3aaf5-147e-439b-b430-8f5d4241b66c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:38.387755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3aaf5-147e-439b-b430-8f5d4241b66c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:38.388258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3aaf5-147e-439b-b430-8f5d4241b66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:38.423681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-22b3aaf5-147e-439b-b430-8f5d4241b66c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:05:38.425575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-22b3aaf5-147e-439b-b430-8f5d4241b66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:38.441344 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1139/2278] 10.222.0.25 () {58 vars in 1016 bytes} [Sun Sep 25 01:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:05:38.470712 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:38.485794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7f166-7da6-4732-bc89-e52461eb56b2 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:38.496575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:38.497792 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:38.498362 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:38.499378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:38.515779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:38.519206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:38.539952 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1140/2279] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:05:47.331618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3aaf5-147e-439b-b430-8f5d4241b66c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:47.346716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3aaf5-147e-439b-b430-8f5d4241b66c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:47.353103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e7404-e5c4-4bcc-8b59-4a77a17e655e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:47.353734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e7404-e5c4-4bcc-8b59-4a77a17e655e service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:47.354014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e7404-e5c4-4bcc-8b59-4a77a17e655e service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:47.354579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e7404-e5c4-4bcc-8b59-4a77a17e655e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:47.357191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a8e7404-e5c4-4bcc-8b59-4a77a17e655e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:47.432709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e7404-e5c4-4bcc-8b59-4a77a17e655e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:47.434348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e7404-e5c4-4bcc-8b59-4a77a17e655e service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:47.445080 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1140/2280] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:05:48.528596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=30 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:57.440171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=46 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:59.771640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:05:59.784148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f96949-9090-49bd-a701-cbff5920db93 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:05:59.790453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:05:59.791128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:05:59.791407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:05:59.791679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:05:59.793949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:05:59.806656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:05:59.808544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:05:59.820617 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1141/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:05:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:06:09.814540 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:15.229487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:15.230311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:15.230311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:15.255640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] MFA Rules not processed for user `ca20e105b76149dd99e4aa64c62eb483`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:06:15.258329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:15.268764 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1141/2282] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:06:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:06:18.094677 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:18.097307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:18.128066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4a1c9-34d7-485f-878a-164285fe3949 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:18.140518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:18.141808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:18.142711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:18.143549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:18.146940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:18.173403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:18.177079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:18.196980 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1142/2283] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:06:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:06:18.254951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:18.258528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:18.279103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37aa60e-5ea6-43f0-9422-1818e9e7775a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:18.285213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:18.286162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:18.286635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:18.287109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] PATH_INFO: `/v3/users/2413fa59e75741a68f391e55a8a7dc2e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:18.288662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2413fa59e75741a68f391e55a8a7dc2e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:18.290682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:18.375676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2413fa59e75741a68f391e55a8a7dc2e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.376508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2413fa59e75741a68f391e55a8a7dc2e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.386142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2413fa59e75741a68f391e55a8a7dc2e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.401550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2413fa59e75741a68f391e55a8a7dc2e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.402710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-860502bd-a7e8-49a5-a66c-b1564998f3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2413fa59e75741a68f391e55a8a7dc2e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.422120 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1142/2284] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:06:18 2022] DELETE /identity/v3/users/2413fa59e75741a68f391e55a8a7dc2e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:06:18.647725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:18.663528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bcaa3-2e73-4ec1-802e-94fc8936d577 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:18.745380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:18.746021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:18.746307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:18.746584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] PATH_INFO: `/v3/projects/f53d77003a86430b93638b729930f788` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:18.747866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f53d77003a86430b93638b729930f788)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:18.749882 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:18.822624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f53d77003a86430b93638b729930f788', 'request_id': 'req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.830411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f53d77003a86430b93638b729930f788', 'request_id': 'req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.838214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f53d77003a86430b93638b729930f788', 'request_id': 'req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.854006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] The token cache is being invalidate because project f53d77003a86430b93638b729930f788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:06:18.854321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.855046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7fb4f7c-17d8-4e9b-9e7b-4d9d7d322a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:18.856889 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1143/2285] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:06:18 2022] DELETE /identity/v3/projects/f53d77003a86430b93638b729930f788 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:06:18.862151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c6914-bd1f-4516-ad1d-50426605e0ba None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:18.862526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c6914-bd1f-4516-ad1d-50426605e0ba None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:18.862865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c6914-bd1f-4516-ad1d-50426605e0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:18.885413 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-805c6914-bd1f-4516-ad1d-50426605e0ba None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:06:18.886907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-805c6914-bd1f-4516-ad1d-50426605e0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:18.959130 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1143/2286] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:06:18.967330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe6c4-5415-493d-8820-1fb954cedc1d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:18.967606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe6c4-5415-493d-8820-1fb954cedc1d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:18.967880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe6c4-5415-493d-8820-1fb954cedc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:18.989250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb6fe6c4-5415-493d-8820-1fb954cedc1d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:06:18.990480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb6fe6c4-5415-493d-8820-1fb954cedc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:18.998820 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1144/2287] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:06:19.003379 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c6914-bd1f-4516-ad1d-50426605e0ba None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:19.015259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c6914-bd1f-4516-ad1d-50426605e0ba None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:19.021580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:19.022264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.022666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.022983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.023702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:06:19.024052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:19.025857 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:19.040742 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1144/2288] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:06:19.045697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fe6c4-5415-493d-8820-1fb954cedc1d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:19.056711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fe6c4-5415-493d-8820-1fb954cedc1d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:19.062232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:19.062872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.063147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.063461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.064275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:19.066648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:19.100188 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1145/2289] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:06:19 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:06:19.105102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:19.116154 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da50b-7db2-4330-93b5-2dfdbcf9d0dd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:19.175939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:19.191728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:19.192685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.193101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.193468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.194234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:19.197420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:19.222159 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1145/2290] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:06:19 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:06:19.227794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:19.239746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0bd70-debe-4a45-b276-83dddc9d829d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:19.244816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:19.245448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.245733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.246033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.246697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:19.248331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:19.262037 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1146/2291] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:06:19.267194 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:19.278932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935aeb7-0b58-4f72-98bb-ae6e5f849390 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:19.290687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a0e59-29df-493f-938a-3fa9d3f38e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:19.291571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0e59-29df-493f-938a-3fa9d3f38e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.291887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0e59-29df-493f-938a-3fa9d3f38e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.292307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a0e59-29df-493f-938a-3fa9d3f38e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.293057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a0e59-29df-493f-938a-3fa9d3f38e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:19.294691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a0e59-29df-493f-938a-3fa9d3f38e14 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:19.307390 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1146/2292] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:06:19.312100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:19.323622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315055a4-6b49-4432-9f21-fff662709c82 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:19.329062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:19.329805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.330090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.330372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] PATH_INFO: `/v3/projects/b0537792603544baa240851a8a6a5b08/users/8a05f9d972b545b8af1963f8bb7d01eb/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.357131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0537792603544baa240851a8a6a5b08, user_id=8a05f9d972b545b8af1963f8bb7d01eb, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:19.361735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:19.381219 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1147/2293] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:06:19 2022] PUT /identity/v3/projects/b0537792603544baa240851a8a6a5b08/users/8a05f9d972b545b8af1963f8bb7d01eb/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:06:19.393277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.393576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.393817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.430192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] MFA Rules not processed for user `8a05f9d972b545b8af1963f8bb7d01eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:06:19.431601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:19.543845 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1147/2294] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:06:19 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:06:19.556641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:19.559917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:19.583348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:19.658126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9dfe-1c24-4d07-8c23-948e28bf4e0e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:19.662700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:19.663917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:19.664207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:19.664521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:19.674841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:19.676412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:19.687709 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1148/2295] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:06:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:06:22.745237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:22.747322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:22.854944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f31d4-ded1-44fa-a049-0f92b8d4bf79 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:22.859291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2dd3ba-7ead-427c-8996-ae5d46f2e531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:22.860153 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dd3ba-7ead-427c-8996-ae5d46f2e531 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:22.860488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dd3ba-7ead-427c-8996-ae5d46f2e531 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:22.860822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dd3ba-7ead-427c-8996-ae5d46f2e531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:22.863286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3a2dd3ba-7ead-427c-8996-ae5d46f2e531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:22.875673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2dd3ba-7ead-427c-8996-ae5d46f2e531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:22.877197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2dd3ba-7ead-427c-8996-ae5d46f2e531 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:22.888807 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1148/2296] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:06:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:06:25.239903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:25.427858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3b35-48dd-41e0-b7fa-04eadfffeba6 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:25.433861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:25.434563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:25.434845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:25.435116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] PATH_INFO: `/v3/users/8a05f9d972b545b8af1963f8bb7d01eb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:25.436362 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a05f9d972b545b8af1963f8bb7d01eb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:25.438605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:25.509777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a05f9d972b545b8af1963f8bb7d01eb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:25.522803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a05f9d972b545b8af1963f8bb7d01eb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:25.541000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a05f9d972b545b8af1963f8bb7d01eb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:25.542083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a05f9d972b545b8af1963f8bb7d01eb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:25.542897 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a05f9d972b545b8af1963f8bb7d01eb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:25.549033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:25.562400 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1149/2297] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:06:25 2022] DELETE /identity/v3/users/8a05f9d972b545b8af1963f8bb7d01eb => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:06:25.569975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:25.571001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:25.571195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:25.648779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:06:25.650419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:25.693277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:25.730076 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1149/2298] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:06:25.742208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:25.743853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:25.853446 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149df43-0714-49dd-bde0-3f00f1b8e7a0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:25.858555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:25.859229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:25.859537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:25.859933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:25.873358 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:25.875133 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:25.887127 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1150/2299] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:06:26.314024 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:26.316650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:26.337286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f78254-60cc-4be2-8ed4-6ebd6b7c5a91 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:26.344665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:26.345009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:26.345366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:26.345835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] PATH_INFO: `/v3/projects/b0537792603544baa240851a8a6a5b08` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:26.348545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0537792603544baa240851a8a6a5b08)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:26.352264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:26.421328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0537792603544baa240851a8a6a5b08', 'request_id': 'req-876de674-151d-4a9f-9ba6-f834b6f302e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:26.433954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0537792603544baa240851a8a6a5b08', 'request_id': 'req-876de674-151d-4a9f-9ba6-f834b6f302e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:26.443294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0537792603544baa240851a8a6a5b08', 'request_id': 'req-876de674-151d-4a9f-9ba6-f834b6f302e9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:26.458422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] The token cache is being invalidate because project b0537792603544baa240851a8a6a5b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:06:26.458771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:26.459503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:06:26.461547 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1150/2300] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:06:26 2022] DELETE /identity/v3/projects/b0537792603544baa240851a8a6a5b08 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:06:35.881774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=36 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:36.454947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:49.121932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:49.123893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:49.216299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b7cb0-9285-420a-a43a-a5e6899b3724 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:49.321624 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b4008-befd-405d-a4bb-6580319155b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:49.322733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b4008-befd-405d-a4bb-6580319155b7 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:49.323184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b4008-befd-405d-a4bb-6580319155b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:49.323665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b4008-befd-405d-a4bb-6580319155b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:49.327266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-441b4008-befd-405d-a4bb-6580319155b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:49.418231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b4008-befd-405d-a4bb-6580319155b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:49.419993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b4008-befd-405d-a4bb-6580319155b7 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:49.431482 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1151/2301] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:06:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:06:54.601780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:54.604636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:54.629802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876de674-151d-4a9f-9ba6-f834b6f302e9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:54.641798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:54.642487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:54.642816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:54.643089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:54.645191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:54.661581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:54.663365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:54.676127 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1151/2302] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:06:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:06:59.025184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:59.025898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:59.026036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:59.048844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] MFA Rules not processed for user `c468aa6d1248478286fe0a8792411a91`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:06:59.050347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:59.059124 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1152/2303] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:06:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:06:59.468539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:06:59.470273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:59.570411 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f856d2c-8e8e-4ab9-b979-3cd3326d2657 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:06:59.575108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:06:59.575926 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:06:59.576250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:06:59.576605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:06:59.579029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:06:59.591033 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:06:59.592855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:06:59.603910 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1152/2304] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:06:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:07:01.309258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:01.311462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:01.353993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:01.412010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90738f8-3d30-44d9-bb3d-50cfb5005dbf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:01.420852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43dba6-e4d9-4d39-a53d-4ef0bd9bfd76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:01.421658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43dba6-e4d9-4d39-a53d-4ef0bd9bfd76 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:01.421995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43dba6-e4d9-4d39-a53d-4ef0bd9bfd76 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:01.422344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43dba6-e4d9-4d39-a53d-4ef0bd9bfd76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:01.424430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4d43dba6-e4d9-4d39-a53d-4ef0bd9bfd76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:01.439226 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43dba6-e4d9-4d39-a53d-4ef0bd9bfd76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:01.440936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43dba6-e4d9-4d39-a53d-4ef0bd9bfd76 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:01.453744 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1153/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:07:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:07:09.598486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=41 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:11.447717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=19 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:16.095892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:16.101630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:16.132770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8543cc-11f1-497c-a4a2-9bfa0e97be09 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:16.144109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:16.146396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:16.148920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:16.149800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:16.153576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:16.186774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:16.191717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:16.220602 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1153/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:07:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:07:16.322377 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2c975-223e-4970-842d-fa9bea7ae6a0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:16.324231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2c975-223e-4970-842d-fa9bea7ae6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:16.325073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2c975-223e-4970-842d-fa9bea7ae6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:16.365552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-32e2c975-223e-4970-842d-fa9bea7ae6a0 None None] MFA Rules not processed for user `f32c9b07e6234d55b0d4846d44ee6ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:07:16.368037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-32e2c975-223e-4970-842d-fa9bea7ae6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:16.482830 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1154/2307] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:07:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:07:17.045590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:17.047632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:17.061521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea6848-08c4-45dd-a6b5-c0691f2a8ae2 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:17.065734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:17.066371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:17.066650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:17.066923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:17.069449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:17.080328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:17.081828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:17.091431 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1154/2308] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:07:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:07:22.069820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2c975-223e-4970-842d-fa9bea7ae6a0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:22.091176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2c975-223e-4970-842d-fa9bea7ae6a0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:22.106968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ed02c-4f91-4e61-a290-fd3323882221 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:22.107609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed02c-4f91-4e61-a290-fd3323882221 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:22.107891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed02c-4f91-4e61-a290-fd3323882221 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:22.108161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed02c-4f91-4e61-a290-fd3323882221 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:22.112119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-655ed02c-4f91-4e61-a290-fd3323882221 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:22.150106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ed02c-4f91-4e61-a290-fd3323882221 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:22.151766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ed02c-4f91-4e61-a290-fd3323882221 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:22.178633 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1155/2309] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:07:27.087290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=12 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:32.162453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=31 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:36.704023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:36.731326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ec8d-f546-4932-b926-3ab3e6277de3 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:36.742784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:36.743448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:36.743792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:36.744177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:36.747162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:36.905906 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:36.909524 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:36.926327 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1155/2310] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:07:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:07:41.930109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e481354-57fe-484a-9e5a-d9bf72815e2d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:41.930720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e481354-57fe-484a-9e5a-d9bf72815e2d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:41.930720 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e481354-57fe-484a-9e5a-d9bf72815e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:41.981843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1e481354-57fe-484a-9e5a-d9bf72815e2d None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:07:41.983551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1e481354-57fe-484a-9e5a-d9bf72815e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:42.066130 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1156/2311] 10.222.0.26 () {58 vars in 967 bytes} [Sun Sep 25 01:07:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:07:42.134893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:42.136803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:42.232980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9581d-e27b-4453-ac93-e69fce73f49d service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:42.238716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09fd38-4923-4774-bd26-9265cb315f2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:42.239546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09fd38-4923-4774-bd26-9265cb315f2b service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:42.239815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09fd38-4923-4774-bd26-9265cb315f2b service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:42.240086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09fd38-4923-4774-bd26-9265cb315f2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:42.255752 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09fd38-4923-4774-bd26-9265cb315f2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:42.258140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09fd38-4923-4774-bd26-9265cb315f2b service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:42.274061 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1156/2312] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:07:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:07:42.791867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e481354-57fe-484a-9e5a-d9bf72815e2d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:42.805305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e481354-57fe-484a-9e5a-d9bf72815e2d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:42.817748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:42.819141 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:42.819781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:42.820432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:42.846542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:43.006048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:43.009334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:43.027525 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1157/2313] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:07:48.513723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:48.514343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:48.514670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:48.537533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] MFA Rules not processed for user `9d73ba36238f449dbe08d8888920510f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:07:48.539021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:48.547039 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1157/2314] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:07:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:07:50.026175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:50.058485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78a8fa-d1fc-4944-9d23-ac08a5672054 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:50.067835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:50.069053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:50.069566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:50.070006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:50.072685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:50.093939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:50.097143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:50.114400 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1158/2315] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:07:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:07:55.422891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:55.427840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:55.463492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21a18c-5956-4cca-ae95-e09cf4e78151 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:55.473518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:55.475128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:55.475667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:55.476123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] PATH_INFO: `/v3/users/1bc4010eab744cf78053adb904f85dc5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:55.477950 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc4010eab744cf78053adb904f85dc5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:55.480695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:55.567121 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc4010eab744cf78053adb904f85dc5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:55.567875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc4010eab744cf78053adb904f85dc5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:55.577052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc4010eab744cf78053adb904f85dc5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:55.600347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc4010eab744cf78053adb904f85dc5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:55.600347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc4010eab744cf78053adb904f85dc5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:55.603310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:55.621241 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1158/2316] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:07:55 2022] DELETE /identity/v3/users/1bc4010eab744cf78053adb904f85dc5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:07:56.205173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:56.227207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fa636-3a9c-4108-adb0-f8df772bcc95 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:56.339476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:56.340156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:56.340474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:56.340935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] PATH_INFO: `/v3/projects/a49eb5e19c0e44b3b9273f5126c0915f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:56.343992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49eb5e19c0e44b3b9273f5126c0915f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:56.346810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:56.379310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:56.426939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49eb5e19c0e44b3b9273f5126c0915f', 'request_id': 'req-58858d9c-81a8-4eee-9153-c3c79ce8aa62'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.436172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49eb5e19c0e44b3b9273f5126c0915f', 'request_id': 'req-58858d9c-81a8-4eee-9153-c3c79ce8aa62'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.448034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49eb5e19c0e44b3b9273f5126c0915f', 'request_id': 'req-58858d9c-81a8-4eee-9153-c3c79ce8aa62'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.482021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] The token cache is being invalidate because project a49eb5e19c0e44b3b9273f5126c0915f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:07:56.482236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.483613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.487183 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1159/2317] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:07:56 2022] DELETE /identity/v3/projects/a49eb5e19c0e44b3b9273f5126c0915f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:07:56.493131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:56.495541 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:56.513383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25956bcf-e2e2-4296-9e99-06d5c64e53be admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:56.606600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:56.607259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:56.607548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:56.607812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] PATH_INFO: `/v3/users/3c5febc3151e4d149b49578a39ceeba8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:56.609582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c5febc3151e4d149b49578a39ceeba8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:56.613772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:56.689998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c5febc3151e4d149b49578a39ceeba8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.690941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c5febc3151e4d149b49578a39ceeba8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.700806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c5febc3151e4d149b49578a39ceeba8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.718792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c5febc3151e4d149b49578a39ceeba8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.719738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c5febc3151e4d149b49578a39ceeba8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:56.738830 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1159/2318] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:07:56 2022] DELETE /identity/v3/users/3c5febc3151e4d149b49578a39ceeba8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:07:57.135373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:07:57.137087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:07:57.157840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58858d9c-81a8-4eee-9153-c3c79ce8aa62 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:07:57.273161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:07:57.273847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:07:57.274085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:07:57.274643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] PATH_INFO: `/v3/projects/e694c5b3e92841958444552023c7d701` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:07:57.276095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e694c5b3e92841958444552023c7d701)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:07:57.278243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:07:57.361768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e694c5b3e92841958444552023c7d701', 'request_id': 'req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:57.384137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e694c5b3e92841958444552023c7d701', 'request_id': 'req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:57.401744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e694c5b3e92841958444552023c7d701', 'request_id': 'req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:57.435608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] The token cache is being invalidate because project e694c5b3e92841958444552023c7d701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:07:57.436279 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:57.437194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:07:57.441410 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1160/2319] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:07:57 2022] DELETE /identity/v3/projects/e694c5b3e92841958444552023c7d701 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:08:01.137635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:01.139481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:01.219907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b77e77-8e06-4000-8f6e-7497268e895c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:01.261599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:01.308781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:01.309550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:01.309936 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:01.310675 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:01.312955 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:01.435956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:01.438780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:01.458685 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1160/2320] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:08:05.631178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:05.633472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:05.655431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf1b7e-f2f9-4eaa-b3bf-c59d6f456e1e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:05.661184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:05.661845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:05.662146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:05.662399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] PATH_INFO: `/v3/users/ca20e105b76149dd99e4aa64c62eb483` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:05.663575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca20e105b76149dd99e4aa64c62eb483)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:05.665976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:05.708245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:05.724147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca20e105b76149dd99e4aa64c62eb483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:05.732834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca20e105b76149dd99e4aa64c62eb483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:05.748122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca20e105b76149dd99e4aa64c62eb483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:05.748741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca20e105b76149dd99e4aa64c62eb483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:05.749497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfc92e0b-4824-446b-b8dd-515d942bb1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca20e105b76149dd99e4aa64c62eb483'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:05.762941 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1161/2321] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:08:05 2022] DELETE /identity/v3/users/ca20e105b76149dd99e4aa64c62eb483 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:05.990781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:06.005886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853c350-b51e-4198-9848-e16a05e2c45b service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:06.084210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:06.084933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.085226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.085638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] PATH_INFO: `/v3/projects/5b11c2ffde89471cafaf35f8579e3ab2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.086971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b11c2ffde89471cafaf35f8579e3ab2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:06.089064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:06.153748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b11c2ffde89471cafaf35f8579e3ab2', 'request_id': 'req-dfca10bb-59bb-4f71-b224-a9e4198ea240'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:06.162673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b11c2ffde89471cafaf35f8579e3ab2', 'request_id': 'req-dfca10bb-59bb-4f71-b224-a9e4198ea240'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:06.169837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b11c2ffde89471cafaf35f8579e3ab2', 'request_id': 'req-dfca10bb-59bb-4f71-b224-a9e4198ea240'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:06.182932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] The token cache is being invalidate because project 5b11c2ffde89471cafaf35f8579e3ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:08:06.183228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:06.183911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfca10bb-59bb-4f71-b224-a9e4198ea240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:06.185459 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1161/2322] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:08:05 2022] DELETE /identity/v3/projects/5b11c2ffde89471cafaf35f8579e3ab2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:06.191653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944e09-ed58-4025-829a-2c313e0cde24 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.191951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944e09-ed58-4025-829a-2c313e0cde24 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.192228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944e09-ed58-4025-829a-2c313e0cde24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.254384 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7b944e09-ed58-4025-829a-2c313e0cde24 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:06.255663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7b944e09-ed58-4025-829a-2c313e0cde24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:06.327702 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1162/2323] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:08:06.336466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f5e7-adc3-4588-bf66-1bc8b4052619 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.337060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f5e7-adc3-4588-bf66-1bc8b4052619 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.337406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f5e7-adc3-4588-bf66-1bc8b4052619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.375040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0109f5e7-adc3-4588-bf66-1bc8b4052619 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:06.378197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0109f5e7-adc3-4588-bf66-1bc8b4052619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:06.393304 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1162/2324] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:06.399660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944e09-ed58-4025-829a-2c313e0cde24 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:06.410756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944e09-ed58-4025-829a-2c313e0cde24 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:06.419259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:06.420391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.421094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.421627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.422828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:08:06.423381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:06.426772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:06.449677 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1163/2325] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:08:06.456414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109f5e7-adc3-4588-bf66-1bc8b4052619 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:06.461543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:06.467661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109f5e7-adc3-4588-bf66-1bc8b4052619 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:06.477659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:06.479027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.479027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.479027 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.479956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:06.483469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:06.553373 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1163/2326] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:08:06 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:08:06.558120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:06.572853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb0b77-e704-4ee3-a106-b2cba2ebf171 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:06.664023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:06.665071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.665071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.665666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.667309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:06.671363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:06.706820 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1164/2327] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:08:06 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:08:06.711755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:06.729166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e3e84-6506-4533-8c48-8604244719d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:06.735128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:06.735501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.735805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.736050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.741098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:06.743205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:06.757032 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1164/2328] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:08:06.761298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:06.767079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:06.774791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0bf99-abb4-41e7-8997-2354881133b6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:06.781250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b6610-f64d-4fdc-b8e9-60a0125a136a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:06.782769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b6610-f64d-4fdc-b8e9-60a0125a136a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.783148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b6610-f64d-4fdc-b8e9-60a0125a136a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.783645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b6610-f64d-4fdc-b8e9-60a0125a136a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.784426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b6610-f64d-4fdc-b8e9-60a0125a136a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:06.786893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b6610-f64d-4fdc-b8e9-60a0125a136a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:06.807623 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1165/2329] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:08:06.811932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:06.823151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd2146-7ab4-4407-b2d8-9986f74abad4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:06.830097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:06.831504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.831785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.832109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] PATH_INFO: `/v3/projects/45d8a0029ca14f478abe3a7fc6c9ce2c/users/e600e49531e14a4ab1bb0a213c8a2000/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.856034 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d8a0029ca14f478abe3a7fc6c9ce2c, user_id=e600e49531e14a4ab1bb0a213c8a2000, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:06.859032 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:06.876323 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1165/2330] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:08:06 2022] PUT /identity/v3/projects/45d8a0029ca14f478abe3a7fc6c9ce2c/users/e600e49531e14a4ab1bb0a213c8a2000/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:06.887839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3d299-0518-46ef-865e-651cd1b411b6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:06.888221 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3d299-0518-46ef-865e-651cd1b411b6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:06.888601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3d299-0518-46ef-865e-651cd1b411b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:06.925678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-00d3d299-0518-46ef-865e-651cd1b411b6 None None] MFA Rules not processed for user `e600e49531e14a4ab1bb0a213c8a2000`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:06.927933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-00d3d299-0518-46ef-865e-651cd1b411b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:07.020293 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1166/2331] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:07.026435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.114963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580261-4149-40f3-95c4-b571b56c7836 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.120591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.121222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.121519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.121794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.122482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.124569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:07.159468 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1166/2332] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:08:07 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:08:07.166341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3d299-0518-46ef-865e-651cd1b411b6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.179995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3d299-0518-46ef-865e-651cd1b411b6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.268228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.269409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.270187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.270392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.271544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.275410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:07.320435 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1167/2333] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:08:07 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:08:07.321867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.333847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423cbf9-2a93-4d13-b2f5-e3816da4b56a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.345201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.346134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.346816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.347093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.347780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.350396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:07.363643 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1167/2334] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:08:07.369083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.383162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6e048-aeb2-4db3-9c84-c9fd3cd1427c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.388807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519f7a9-e31e-401c-935f-fb061d3244a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.389564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519f7a9-e31e-401c-935f-fb061d3244a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.389825 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519f7a9-e31e-401c-935f-fb061d3244a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.390101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519f7a9-e31e-401c-935f-fb061d3244a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.390780 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519f7a9-e31e-401c-935f-fb061d3244a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.392493 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519f7a9-e31e-401c-935f-fb061d3244a5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:07.406025 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1168/2335] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:08:07.410976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.422300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c8ec2-2439-4ca0-bc18-fecba7c468e8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.427953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.429329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.429329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.429329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] PATH_INFO: `/v3/projects/3ae1932c039e465f8e65928231570f48/users/cee587e50ecb4f1fb0692fb5683ccdf5/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.449333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae1932c039e465f8e65928231570f48, user_id=cee587e50ecb4f1fb0692fb5683ccdf5, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.452603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:07.473112 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1168/2336] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:08:07 2022] PUT /identity/v3/projects/3ae1932c039e465f8e65928231570f48/users/cee587e50ecb4f1fb0692fb5683ccdf5/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:07.486510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f31887-1ac6-4669-ab00-3ed07dec556e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.486934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f31887-1ac6-4669-ab00-3ed07dec556e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.487283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f31887-1ac6-4669-ab00-3ed07dec556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.524870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-00f31887-1ac6-4669-ab00-3ed07dec556e None None] MFA Rules not processed for user `cee587e50ecb4f1fb0692fb5683ccdf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:07.526084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-00f31887-1ac6-4669-ab00-3ed07dec556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:07.587344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:07.623738 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1169/2337] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:07.630669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.704837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:07.731104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18b8b1-5b54-45c7-965d-8eb51edcbebd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.739807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.740892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.741381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.742049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.743084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.746981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:07.792153 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1169/2338] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:08:07 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:08:07.797162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f31887-1ac6-4669-ab00-3ed07dec556e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.810547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f31887-1ac6-4669-ab00-3ed07dec556e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.902707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.903462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.903724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.904092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.904820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.907068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:07.941009 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1170/2339] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:08:07 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:08:07.943407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:07.963976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabb106-e02f-40ff-8cee-a73ef0429e7c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:07.972999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:07.974129 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:07.974606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:07.975087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:07.976290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:07.980257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:08.009770 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1170/2340] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:08:08.015312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:08.033786 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c409ad5-7970-4697-b041-02346441aff8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:08.039698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:08.040754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:08.041100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:08.041409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] PATH_INFO: `/v3/projects/69d02d333c714c6e8ce9a7f43f469737/users/4ffa71aa8d4f4796a21a9309a69a1c83/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:08.063677 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d02d333c714c6e8ce9a7f43f469737, user_id=4ffa71aa8d4f4796a21a9309a69a1c83, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:08.067316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:08.085058 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1171/2341] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:08:08 2022] PUT /identity/v3/projects/69d02d333c714c6e8ce9a7f43f469737/users/4ffa71aa8d4f4796a21a9309a69a1c83/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:08.089445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:08.101414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417acbb-4866-4986-94ee-9c8602faaaf9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:08.194662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd6443-222e-4480-8a3b-fc54331475ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:08.195559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd6443-222e-4480-8a3b-fc54331475ae admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:08.195855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd6443-222e-4480-8a3b-fc54331475ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:08.196284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd6443-222e-4480-8a3b-fc54331475ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:08.196993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd6443-222e-4480-8a3b-fc54331475ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:08.198652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd6443-222e-4480-8a3b-fc54331475ae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:08.213957 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1171/2342] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:08:08.220474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:08.244414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393bd35-8a94-436b-8a0d-959668d09d8b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:08.254245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:08.255570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:08.256426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:08.257411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] PATH_INFO: `/v3/projects/69d02d333c714c6e8ce9a7f43f469737/users/4ffa71aa8d4f4796a21a9309a69a1c83/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:08.262471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d02d333c714c6e8ce9a7f43f469737, user_id=4ffa71aa8d4f4796a21a9309a69a1c83, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:08.274403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:08.292430 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1172/2343] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:08:08 2022] PUT /identity/v3/projects/69d02d333c714c6e8ce9a7f43f469737/users/4ffa71aa8d4f4796a21a9309a69a1c83/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:08.309515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:08.310038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:08.310038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:08.349240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] MFA Rules not processed for user `4ffa71aa8d4f4796a21a9309a69a1c83`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:08.351735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:08.469417 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1172/2344] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:08.483424 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:08.487267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:08.622170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365580c-9cf3-44f7-a9f8-9aa8603f8d68 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:08.631108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:08.632326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:08.636259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:08.636259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:08.652236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:08.654978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:08.674691 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1173/2345] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:08:08.741715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:08.744224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:08.762220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91bb5e-4dbb-4bd6-96d6-1eca6b4fa0ed None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:08.768886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05270a6-ae85-4bd7-b3fb-8c03cc05dd69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:08.770764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05270a6-ae85-4bd7-b3fb-8c03cc05dd69 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:08.771850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05270a6-ae85-4bd7-b3fb-8c03cc05dd69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:08.772354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05270a6-ae85-4bd7-b3fb-8c03cc05dd69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:08.850007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:08.875775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05270a6-ae85-4bd7-b3fb-8c03cc05dd69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:08.878172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05270a6-ae85-4bd7-b3fb-8c03cc05dd69 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:08.889692 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1173/2346] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:08:12.135221 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:12.168149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:12.228193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80cd2f-27b9-48e9-90f2-3fc66a43fbcf service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:12.233713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:12.234533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:12.234868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:12.235321 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] PATH_INFO: `/v3/users/e600e49531e14a4ab1bb0a213c8a2000` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:12.237029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e600e49531e14a4ab1bb0a213c8a2000)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:12.239245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:12.324245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e600e49531e14a4ab1bb0a213c8a2000'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:12.344778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e600e49531e14a4ab1bb0a213c8a2000'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:12.364589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e600e49531e14a4ab1bb0a213c8a2000'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:12.365275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e600e49531e14a4ab1bb0a213c8a2000'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:12.365943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e600e49531e14a4ab1bb0a213c8a2000'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:12.386259 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1174/2347] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:08:12 2022] DELETE /identity/v3/users/e600e49531e14a4ab1bb0a213c8a2000 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:12.392866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:12.393762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:12.393762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:12.416998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:12.418268 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:12.497922 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1174/2348] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:12.509985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:12.511498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:12.583158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da821f-7471-4d40-9dd1-d09daf0083f7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:12.588338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:12.589106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:12.589462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:12.589814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:12.601008 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:12.602730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:12.614759 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1175/2349] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:08:13.042105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:13.044104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:13.064679 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20313f77-1a86-4156-b705-bc3c8ad44dfb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:13.072528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:13.073778 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:13.074143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:13.074493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] PATH_INFO: `/v3/projects/45d8a0029ca14f478abe3a7fc6c9ce2c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:13.076223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d8a0029ca14f478abe3a7fc6c9ce2c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:13.078785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:13.151161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d8a0029ca14f478abe3a7fc6c9ce2c', 'request_id': 'req-9476bbb8-5368-40bd-808a-54d4307be7dc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.163591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d8a0029ca14f478abe3a7fc6c9ce2c', 'request_id': 'req-9476bbb8-5368-40bd-808a-54d4307be7dc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.171464 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d8a0029ca14f478abe3a7fc6c9ce2c', 'request_id': 'req-9476bbb8-5368-40bd-808a-54d4307be7dc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.185579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] The token cache is being invalidate because project 45d8a0029ca14f478abe3a7fc6c9ce2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:08:13.185800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.186518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.188028 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1175/2350] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:08:13 2022] DELETE /identity/v3/projects/45d8a0029ca14f478abe3a7fc6c9ce2c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:13.193269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:13.209884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae7399-a0f6-43b4-abc5-148e14530db5 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:13.285741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:13.311718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:13.311718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:13.311718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:13.311718 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] PATH_INFO: `/v3/users/cee587e50ecb4f1fb0692fb5683ccdf5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:13.312526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee587e50ecb4f1fb0692fb5683ccdf5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:13.312526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:13.386660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee587e50ecb4f1fb0692fb5683ccdf5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.400458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee587e50ecb4f1fb0692fb5683ccdf5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.420346 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee587e50ecb4f1fb0692fb5683ccdf5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.421223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee587e50ecb4f1fb0692fb5683ccdf5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.425389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee587e50ecb4f1fb0692fb5683ccdf5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:13.448142 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1176/2351] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:08:13 2022] DELETE /identity/v3/users/cee587e50ecb4f1fb0692fb5683ccdf5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:13.924236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:13.926882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:13.950795 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476bbb8-5368-40bd-808a-54d4307be7dc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:14.051965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:14.052610 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:14.052941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:14.053263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] PATH_INFO: `/v3/projects/3ae1932c039e465f8e65928231570f48` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:14.055084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae1932c039e465f8e65928231570f48)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:14.057229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:14.065074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:14.129961 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae1932c039e465f8e65928231570f48', 'request_id': 'req-0668a9bb-a65d-409d-a5ee-a49c04966412'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.141500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae1932c039e465f8e65928231570f48', 'request_id': 'req-0668a9bb-a65d-409d-a5ee-a49c04966412'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.148862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae1932c039e465f8e65928231570f48', 'request_id': 'req-0668a9bb-a65d-409d-a5ee-a49c04966412'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.162619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] The token cache is being invalidate because project 3ae1932c039e465f8e65928231570f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:08:14.162948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.163700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.165626 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1176/2352] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:08:13 2022] DELETE /identity/v3/projects/3ae1932c039e465f8e65928231570f48 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:14.170912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:14.173335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:14.198036 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe606847-7d90-4254-a0f7-02c91993cbca admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:14.292651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:14.293335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:14.293620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:14.293928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] PATH_INFO: `/v3/users/4ffa71aa8d4f4796a21a9309a69a1c83` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:14.295413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ffa71aa8d4f4796a21a9309a69a1c83)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:14.297756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:14.382157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ffa71aa8d4f4796a21a9309a69a1c83'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.393665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ffa71aa8d4f4796a21a9309a69a1c83'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.407931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:14.411076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ffa71aa8d4f4796a21a9309a69a1c83'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.412053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ffa71aa8d4f4796a21a9309a69a1c83'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.413549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-367f5b31-ff1f-40c0-9be4-f55eb4ce9b5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ffa71aa8d4f4796a21a9309a69a1c83'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:14.432071 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1177/2353] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:08:14 2022] DELETE /identity/v3/users/4ffa71aa8d4f4796a21a9309a69a1c83 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:14.822968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:14.825640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:14.850407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668a9bb-a65d-409d-a5ee-a49c04966412 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:14.935159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:14.936253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:14.936253 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:14.936593 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] PATH_INFO: `/v3/projects/69d02d333c714c6e8ce9a7f43f469737` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:14.937817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d02d333c714c6e8ce9a7f43f469737)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:14.939972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.005084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d02d333c714c6e8ce9a7f43f469737', 'request_id': 'req-163aae80-1ca5-4898-b405-075b26f1e99e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:15.015839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d02d333c714c6e8ce9a7f43f469737', 'request_id': 'req-163aae80-1ca5-4898-b405-075b26f1e99e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:15.024809 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d02d333c714c6e8ce9a7f43f469737', 'request_id': 'req-163aae80-1ca5-4898-b405-075b26f1e99e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:15.038612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] The token cache is being invalidate because project 69d02d333c714c6e8ce9a7f43f469737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:08:15.039041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:15.040232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-163aae80-1ca5-4898-b405-075b26f1e99e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:15.042332 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1177/2354] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:08:14 2022] DELETE /identity/v3/projects/69d02d333c714c6e8ce9a7f43f469737 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:15.050077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa6ac-8312-4a12-bc8c-39007f4fdaa8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.050371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa6ac-8312-4a12-bc8c-39007f4fdaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.050638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa6ac-8312-4a12-bc8c-39007f4fdaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.075645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ccafa6ac-8312-4a12-bc8c-39007f4fdaa8 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:15.077837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ccafa6ac-8312-4a12-bc8c-39007f4fdaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:15.186435 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1178/2355] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:15.198612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140a124-3f2a-4878-ba67-22f2971ac8b0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.199000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140a124-3f2a-4878-ba67-22f2971ac8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.199509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140a124-3f2a-4878-ba67-22f2971ac8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.224478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8140a124-3f2a-4878-ba67-22f2971ac8b0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:15.226633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8140a124-3f2a-4878-ba67-22f2971ac8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:15.241334 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1178/2356] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:15.246174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafa6ac-8312-4a12-bc8c-39007f4fdaa8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:15.258315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafa6ac-8312-4a12-bc8c-39007f4fdaa8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:15.263678 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:15.264360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.264725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.265043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.265821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:08:15.266222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:15.268611 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.282629 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1179/2357] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:08:15.290208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140a124-3f2a-4878-ba67-22f2971ac8b0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:15.301276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140a124-3f2a-4878-ba67-22f2971ac8b0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:15.308130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:15.309093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.309505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.309913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.310902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:15.313366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.331949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:15.352422 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1179/2358] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:08:15 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:08:15.354517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:15.372957 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2b3e3-94b6-47db-84ac-49b1ea729cb3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:15.450268 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:15.450818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.451691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.452095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.452912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:15.454991 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.480903 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1180/2359] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:08:15 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:08:15.485554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:15.498609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566ddf8-cf1a-4552-b0da-7346cbfb7c79 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:15.504596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:15.505315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.505632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.505945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.506640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:15.508438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.520884 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1180/2360] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:08:15.526658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:15.538123 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71760ce-d047-4c2d-ab04-5eb0bd0e4b3a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:15.543237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be0e9e-61f0-47ee-8362-2d9e9a1af3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:15.543863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be0e9e-61f0-47ee-8362-2d9e9a1af3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.544148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be0e9e-61f0-47ee-8362-2d9e9a1af3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.544802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be0e9e-61f0-47ee-8362-2d9e9a1af3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.545120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be0e9e-61f0-47ee-8362-2d9e9a1af3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:15.546864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be0e9e-61f0-47ee-8362-2d9e9a1af3ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.560652 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1181/2361] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:08:15.565927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:15.584271 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ceeed-3eb1-4052-8fb1-04f2e9c8e1e1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:15.592527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:15.593627 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.594293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.594574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] PATH_INFO: `/v3/projects/37c3708702654977b6b7bbb104255e3f/users/d584320fc5044638914728ab9d050fc4/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.622948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c3708702654977b6b7bbb104255e3f, user_id=d584320fc5044638914728ab9d050fc4, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:15.628302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.652073 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1181/2362] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:08:15 2022] PUT /identity/v3/projects/37c3708702654977b6b7bbb104255e3f/users/d584320fc5044638914728ab9d050fc4/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:08:15.658615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2010-eced-427c-9d7a-767c150e1884 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.658930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2010-eced-427c-9d7a-767c150e1884 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.659201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2010-eced-427c-9d7a-767c150e1884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.680547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c0ab2010-eced-427c-9d7a-767c150e1884 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:15.682165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c0ab2010-eced-427c-9d7a-767c150e1884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:15.739250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:15.761582 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1182/2363] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:08:15.773036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:15.775026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:15.846149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba4d95-e6f6-4feb-b409-8b1bf5bbcc87 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:15.850136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:15.850755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:15.851037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:15.851311 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:15.863001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:15.864535 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:15.876088 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1182/2364] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:08:16.729714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaf808-1e2a-4933-97bd-3fefdde22805 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:16.730280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaf808-1e2a-4933-97bd-3fefdde22805 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:16.730877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaf808-1e2a-4933-97bd-3fefdde22805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:16.774927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4ceaf808-1e2a-4933-97bd-3fefdde22805 None None] MFA Rules not processed for user `d584320fc5044638914728ab9d050fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:16.777175 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4ceaf808-1e2a-4933-97bd-3fefdde22805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:16.900807 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1183/2365] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:16.906091 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:16.926420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba920a-7024-4fb4-af0c-01db64a3503a service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:16.934858 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:16.935914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:16.936396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:16.936976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:16.938222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:16.943042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:16.996468 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1183/2366] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:08:16 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:08:17.001416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaf808-1e2a-4933-97bd-3fefdde22805 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:17.025690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaf808-1e2a-4933-97bd-3fefdde22805 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:17.122789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:17.123996 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:17.124403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:17.124797 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:17.125660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:17.128488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:17.155438 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1184/2367] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:08:16 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:08:17.160237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:17.172942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec551c-fba0-4cee-b02d-91966a5264c1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:17.179269 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:17.179894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:17.180200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:17.180472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:17.181123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:17.182730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:17.195875 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1184/2368] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:08:17.201124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:17.215876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe2a79-81fe-43b6-943f-5afbe8377559 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:17.223138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:17.224098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:17.224550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:17.224962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] PATH_INFO: `/v3/projects/f956ac3dbb8e4630a983d8dfdd3b78be/users/a7d49b7b7e6540cbbc8413d2d99bdb04/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:17.250229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f956ac3dbb8e4630a983d8dfdd3b78be, user_id=a7d49b7b7e6540cbbc8413d2d99bdb04, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:17.253982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:17.272250 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1185/2369] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:08:17 2022] PUT /identity/v3/projects/f956ac3dbb8e4630a983d8dfdd3b78be/users/a7d49b7b7e6540cbbc8413d2d99bdb04/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:17.276942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:17.291640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921902-f21b-446c-887d-77674b106ee4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:17.375659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9c262-ba97-4c83-94d5-de0646921052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:17.376316 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9c262-ba97-4c83-94d5-de0646921052 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:17.376830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9c262-ba97-4c83-94d5-de0646921052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:17.377193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9c262-ba97-4c83-94d5-de0646921052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:17.377968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9c262-ba97-4c83-94d5-de0646921052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:17.380485 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9c262-ba97-4c83-94d5-de0646921052 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:17.393015 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1185/2370] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:08:17.397650 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:17.409949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b82e-03ef-4c3f-94e0-7762934662dd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:17.420280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44909789-8317-49f3-9523-ca61f3ce0e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:17.421582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44909789-8317-49f3-9523-ca61f3ce0e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:17.421936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44909789-8317-49f3-9523-ca61f3ce0e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:17.422413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44909789-8317-49f3-9523-ca61f3ce0e8e admin admin] PATH_INFO: `/v3/projects/f956ac3dbb8e4630a983d8dfdd3b78be/users/a7d49b7b7e6540cbbc8413d2d99bdb04/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:17.426330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44909789-8317-49f3-9523-ca61f3ce0e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f956ac3dbb8e4630a983d8dfdd3b78be, user_id=a7d49b7b7e6540cbbc8413d2d99bdb04, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:17.431487 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44909789-8317-49f3-9523-ca61f3ce0e8e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:17.445576 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1186/2371] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:08:17 2022] PUT /identity/v3/projects/f956ac3dbb8e4630a983d8dfdd3b78be/users/a7d49b7b7e6540cbbc8413d2d99bdb04/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:08:18.419724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:18.420676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:18.420976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:18.461943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] MFA Rules not processed for user `a7d49b7b7e6540cbbc8413d2d99bdb04`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:18.464812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:18.490262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:18.595439 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1186/2372] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:18.608204 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:18.608865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:18.608865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:18.632773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] MFA Rules not processed for user `d584320fc5044638914728ab9d050fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:18.635145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:18.691824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:18.738255 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1187/2373] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:08:18.754445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:18.756426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:18.919280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a41308-5684-4579-a543-b6c112b3c3b1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:18.926519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:18.927894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:18.928276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:18.928720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:18.942836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:18.946530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:18.958351 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1187/2374] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:08:20.639399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:20.641002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:20.780736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e1694-db8b-49ba-83f1-83c185935f26 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:20.787000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:20.787881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:20.788265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:20.788756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:20.791485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:20.806090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:20.807814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:20.820004 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1188/2375] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:08:28.953685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=68 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:30.814947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=51 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:34.880743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:34.882573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:34.958449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9200-52ec-4aab-ae8b-df5bc82fe566 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:34.962511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:34.963149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:34.963433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:34.963714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:34.975243 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:34.976889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:34.988661 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1188/2376] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:08:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:08:44.982996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=26 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:58.394520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:58.514076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dcc78-0ce8-4cf2-9fa1-c0edc5c38d7d service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:58.519980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:58.520999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:58.524322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:58.524322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] PATH_INFO: `/v3/users/d584320fc5044638914728ab9d050fc4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:58.524322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d584320fc5044638914728ab9d050fc4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:58.525761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:58.599265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd584320fc5044638914728ab9d050fc4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:58.610009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd584320fc5044638914728ab9d050fc4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:58.629400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd584320fc5044638914728ab9d050fc4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:58.630291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd584320fc5044638914728ab9d050fc4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:58.631022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd584320fc5044638914728ab9d050fc4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:58.645921 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1189/2377] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:08:58 2022] DELETE /identity/v3/users/d584320fc5044638914728ab9d050fc4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:58.850374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:58.869528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea989f-604f-4a60-aa7a-e58d9ed5042b service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:58.945000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:58.945997 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:58.946451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:58.946916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] PATH_INFO: `/v3/projects/37c3708702654977b6b7bbb104255e3f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:58.948683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c3708702654977b6b7bbb104255e3f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:58.951807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:59.021081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c3708702654977b6b7bbb104255e3f', 'request_id': 'req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.032134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c3708702654977b6b7bbb104255e3f', 'request_id': 'req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.040310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c3708702654977b6b7bbb104255e3f', 'request_id': 'req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.055989 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] The token cache is being invalidate because project 37c3708702654977b6b7bbb104255e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:08:59.056284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.057106 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.058638 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1189/2378] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:08:58 2022] DELETE /identity/v3/projects/37c3708702654977b6b7bbb104255e3f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:59.063626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:59.064392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:59.079034 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ebd5e-bcea-4592-a2f2-82b713f4f609 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:59.150698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:59.151549 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:59.151873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:59.152151 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] PATH_INFO: `/v3/users/a7d49b7b7e6540cbbc8413d2d99bdb04` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:59.153550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d49b7b7e6540cbbc8413d2d99bdb04)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:59.156048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:59.184890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:59.230984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d49b7b7e6540cbbc8413d2d99bdb04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.244950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d49b7b7e6540cbbc8413d2d99bdb04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.260574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d49b7b7e6540cbbc8413d2d99bdb04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.261352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d49b7b7e6540cbbc8413d2d99bdb04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.261987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ecaaf8c-af79-41ed-b89e-48d2528800ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d49b7b7e6540cbbc8413d2d99bdb04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.274935 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1190/2379] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:08:59 2022] DELETE /identity/v3/users/a7d49b7b7e6540cbbc8413d2d99bdb04 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:08:59.491483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:59.493036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:59.510402 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8bfab-ce2f-4576-8fc2-cb9ae459dc9c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:59.592677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:59.593333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:59.593611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:59.593887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] PATH_INFO: `/v3/projects/f956ac3dbb8e4630a983d8dfdd3b78be` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:59.595072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f956ac3dbb8e4630a983d8dfdd3b78be)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:59.597069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:59.642334 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:59.656029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f956ac3dbb8e4630a983d8dfdd3b78be', 'request_id': 'req-e596b22f-d724-45dc-8254-c27f65c575f4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.665346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f956ac3dbb8e4630a983d8dfdd3b78be', 'request_id': 'req-e596b22f-d724-45dc-8254-c27f65c575f4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.672668 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f956ac3dbb8e4630a983d8dfdd3b78be', 'request_id': 'req-e596b22f-d724-45dc-8254-c27f65c575f4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.684880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] The token cache is being invalidate because project f956ac3dbb8e4630a983d8dfdd3b78be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:08:59.685195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.685870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e596b22f-d724-45dc-8254-c27f65c575f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:08:59.689090 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1190/2380] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:08:59 2022] DELETE /identity/v3/projects/f956ac3dbb8e4630a983d8dfdd3b78be => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:08:59.695071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e7b6-e3fa-4e2e-abf7-4b2f98d554b3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:59.695622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e7b6-e3fa-4e2e-abf7-4b2f98d554b3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:59.695622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e7b6-e3fa-4e2e-abf7-4b2f98d554b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:59.716039 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4417e7b6-e3fa-4e2e-abf7-4b2f98d554b3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:59.717308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4417e7b6-e3fa-4e2e-abf7-4b2f98d554b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:59.785550 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1191/2381] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:59.795118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe435f1-db5d-445c-a40b-881ea2326af7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:59.795401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe435f1-db5d-445c-a40b-881ea2326af7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:59.796308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe435f1-db5d-445c-a40b-881ea2326af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:59.817885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fbe435f1-db5d-445c-a40b-881ea2326af7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:08:59.819111 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fbe435f1-db5d-445c-a40b-881ea2326af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:08:59.827536 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1191/2382] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:08:59.832702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417e7b6-e3fa-4e2e-abf7-4b2f98d554b3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:59.843118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417e7b6-e3fa-4e2e-abf7-4b2f98d554b3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:59.847976 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:59.848637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:59.848916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:59.849188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:59.849853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:08:59.850157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:59.851791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:59.865598 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1192/2383] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:08:59.872061 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe435f1-db5d-445c-a40b-881ea2326af7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:59.881852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe435f1-db5d-445c-a40b-881ea2326af7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:08:59.886721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:08:59.887340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:08:59.887613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:08:59.887877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:08:59.888560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:08:59.890548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:08:59.925090 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1192/2384] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:08:59 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:08:59.931397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:08:59.947783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd520a-fc5f-4cb8-a246-e561a7f96272 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:00.036337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:00.036982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:00.037265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:00.037535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:00.038255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:00.040699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:00.064054 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1193/2385] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:08:59 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:00.068272 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:00.079638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86902e-0e60-4a74-87a6-a09245f6d62a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:00.085130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:00.085748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:00.086026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:00.086293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:00.086956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:00.088615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:00.100355 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1193/2386] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:09:00.104615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:00.115734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489a67e-67e0-4d31-9559-038cec755cd0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:00.122910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e0698-1a47-457f-974b-52375ceab5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:00.123539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e0698-1a47-457f-974b-52375ceab5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:00.124140 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e0698-1a47-457f-974b-52375ceab5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:00.124789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e0698-1a47-457f-974b-52375ceab5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:00.126144 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e0698-1a47-457f-974b-52375ceab5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:00.128132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e0698-1a47-457f-974b-52375ceab5d0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:00.140628 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1194/2387] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:00.144935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:00.155924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef877ec-d814-4523-aa1c-06f2d726832f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:00.165309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:00.165991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:00.166266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:00.166625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] PATH_INFO: `/v3/projects/c78143a442af4f3cb94d9e9a33f56b32/users/2ef0e1665dfd4a73ba9d65c391fde13e/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:00.184155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78143a442af4f3cb94d9e9a33f56b32, user_id=2ef0e1665dfd4a73ba9d65c391fde13e, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:00.187319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:00.203803 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1194/2388] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:00 2022] PUT /identity/v3/projects/c78143a442af4f3cb94d9e9a33f56b32/users/2ef0e1665dfd4a73ba9d65c391fde13e/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:00.216284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035f83e-260a-4fff-b11c-b0c592ad3394 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:00.217247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035f83e-260a-4fff-b11c-b0c592ad3394 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:00.217537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035f83e-260a-4fff-b11c-b0c592ad3394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:00.226715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:00.254941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8035f83e-260a-4fff-b11c-b0c592ad3394 None None] MFA Rules not processed for user `2ef0e1665dfd4a73ba9d65c391fde13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:00.256659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8035f83e-260a-4fff-b11c-b0c592ad3394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:00.361232 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1195/2389] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:00.379187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:00.382511 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:00.468768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8a4bd-37df-499b-9b41-f0a2aeb1853b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:00.474716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:00.475364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:00.475832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:00.476303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:00.491872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:00.494687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:00.508447 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1195/2390] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:01.289933 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa9ab5-ddbd-4a25-ab7f-19c57835c3f0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:01.290431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa9ab5-ddbd-4a25-ab7f-19c57835c3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:01.290541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa9ab5-ddbd-4a25-ab7f-19c57835c3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:01.312564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3ffa9ab5-ddbd-4a25-ab7f-19c57835c3f0 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:01.313807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ffa9ab5-ddbd-4a25-ab7f-19c57835c3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:01.379044 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1196/2391] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:01.442321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:01.443918 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:01.528313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:01.540309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f9cb7-881c-405c-85ff-be1bbf32b322 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:01.549943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0a653-bb0b-4716-9cad-1410d9a344f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:01.551019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a653-bb0b-4716-9cad-1410d9a344f3 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:01.551520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a653-bb0b-4716-9cad-1410d9a344f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:01.551982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a653-bb0b-4716-9cad-1410d9a344f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:01.562852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0a653-bb0b-4716-9cad-1410d9a344f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:01.564605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0a653-bb0b-4716-9cad-1410d9a344f3 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:01.574374 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1196/2392] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:09:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:09:06.532405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa9ab5-ddbd-4a25-ab7f-19c57835c3f0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:06.627823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa9ab5-ddbd-4a25-ab7f-19c57835c3f0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:06.632874 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:06.633534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:06.633819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:06.634164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] PATH_INFO: `/v3/users/2ef0e1665dfd4a73ba9d65c391fde13e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:06.635234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef0e1665dfd4a73ba9d65c391fde13e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:06.637379 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:06.708035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:06.717308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef0e1665dfd4a73ba9d65c391fde13e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:06.731194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef0e1665dfd4a73ba9d65c391fde13e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:06.760548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef0e1665dfd4a73ba9d65c391fde13e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:06.764706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef0e1665dfd4a73ba9d65c391fde13e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:06.765493 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef0e1665dfd4a73ba9d65c391fde13e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:06.795742 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1197/2393] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:06 2022] DELETE /identity/v3/users/2ef0e1665dfd4a73ba9d65c391fde13e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:06.801406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:06.801959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:06.802131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:06.829487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:06.830734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:06.930906 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1197/2394] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:06.938916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:06.940648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:07.033811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a8c5f-8be7-4fbe-9350-0057b7863789 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:07.037588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42103061-2651-49e4-8268-d0d375a0ee94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:07.038669 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42103061-2651-49e4-8268-d0d375a0ee94 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:07.038860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42103061-2651-49e4-8268-d0d375a0ee94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:07.038997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42103061-2651-49e4-8268-d0d375a0ee94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:07.051880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42103061-2651-49e4-8268-d0d375a0ee94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:07.055186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42103061-2651-49e4-8268-d0d375a0ee94 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:07.070016 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1198/2395] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:07.482951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:07.486101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:07.510656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8883106-91d6-4d61-9a24-a7a66b671259 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:07.523040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:07.524204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:07.524855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:07.525469 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] PATH_INFO: `/v3/projects/c78143a442af4f3cb94d9e9a33f56b32` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:07.527765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78143a442af4f3cb94d9e9a33f56b32)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:07.531591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:07.615475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78143a442af4f3cb94d9e9a33f56b32', 'request_id': 'req-97b98cd7-f4ca-4162-8059-b3bd7c447345'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:07.629888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78143a442af4f3cb94d9e9a33f56b32', 'request_id': 'req-97b98cd7-f4ca-4162-8059-b3bd7c447345'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:07.641387 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78143a442af4f3cb94d9e9a33f56b32', 'request_id': 'req-97b98cd7-f4ca-4162-8059-b3bd7c447345'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:07.661069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] The token cache is being invalidate because project c78143a442af4f3cb94d9e9a33f56b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:09:07.661944 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:07.662849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97b98cd7-f4ca-4162-8059-b3bd7c447345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:07.665474 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1198/2396] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:09:07 2022] DELETE /identity/v3/projects/c78143a442af4f3cb94d9e9a33f56b32 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:07.670839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad90b-161b-43aa-96bd-3309a1987b4e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:07.671393 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad90b-161b-43aa-96bd-3309a1987b4e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:07.671393 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad90b-161b-43aa-96bd-3309a1987b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:07.698828 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9a0ad90b-161b-43aa-96bd-3309a1987b4e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:07.701350 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a0ad90b-161b-43aa-96bd-3309a1987b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:07.797182 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1199/2397] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:07.805431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad5c30-4932-425c-85b0-d545f4fe24ef None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:07.806082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad5c30-4932-425c-85b0-d545f4fe24ef None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:07.806082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad5c30-4932-425c-85b0-d545f4fe24ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:07.829730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-caad5c30-4932-425c-85b0-d545f4fe24ef None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:07.831101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-caad5c30-4932-425c-85b0-d545f4fe24ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:07.854096 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1199/2398] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:07.862095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ad90b-161b-43aa-96bd-3309a1987b4e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:07.884958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ad90b-161b-43aa-96bd-3309a1987b4e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:07.897282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:07.898384 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:07.899081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:07.899661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:07.901216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:09:07.902338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:07.907942 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:07.937216 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1200/2399] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:09:07.941692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad5c30-4932-425c-85b0-d545f4fe24ef None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:07.961705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad5c30-4932-425c-85b0-d545f4fe24ef None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:07.971277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:07.972337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:07.972848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:07.973319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:07.974491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:07.978198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:08.030444 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1200/2400] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:09:07 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:08.035016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:08.052774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494238b3-84e9-4a19-a89d-2b46d1a4012c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:08.152681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:08.175988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:08.176655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:08.176940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:08.177347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:08.177996 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:08.180027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:08.205636 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1201/2401] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:09:08 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:08.211428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:08.227686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65193eba-0b18-4d47-8af4-e83ed44070bb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:08.235624 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:08.236632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:08.237172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:08.237591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:08.238564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:08.240940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:08.267555 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1201/2402] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:08.269084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:08.287769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cd054-2fdb-4814-aa74-47ea3bf38340 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:08.300756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659645af-4556-463a-9ed8-e03376eabdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:08.301075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659645af-4556-463a-9ed8-e03376eabdbf admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:08.301443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659645af-4556-463a-9ed8-e03376eabdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:08.301889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659645af-4556-463a-9ed8-e03376eabdbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:08.307374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659645af-4556-463a-9ed8-e03376eabdbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:08.309372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659645af-4556-463a-9ed8-e03376eabdbf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:08.332217 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1202/2403] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:08.333848 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:08.351088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54339b-6be1-4ed9-a0eb-d63e21bced59 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:08.358764 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:08.359842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:08.360176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:08.360420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] PATH_INFO: `/v3/projects/07688894b68742acb510b3d05990b60e/users/1a42899c4ae6418b80b299b14906af6d/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:08.383388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07688894b68742acb510b3d05990b60e, user_id=1a42899c4ae6418b80b299b14906af6d, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:08.388180 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:08.412284 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1202/2404] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:08 2022] PUT /identity/v3/projects/07688894b68742acb510b3d05990b60e/users/1a42899c4ae6418b80b299b14906af6d/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:08.425649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420a0af-ac17-488e-a5e2-7dd84e8f62c5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:08.426113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420a0af-ac17-488e-a5e2-7dd84e8f62c5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:08.427413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420a0af-ac17-488e-a5e2-7dd84e8f62c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:08.479772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d420a0af-ac17-488e-a5e2-7dd84e8f62c5 None None] MFA Rules not processed for user `1a42899c4ae6418b80b299b14906af6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:08.482066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d420a0af-ac17-488e-a5e2-7dd84e8f62c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:08.626021 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1203/2405] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:08.639809 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:08.642294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:08.656671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:08.725536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7066302-d84b-4bb3-8e68-2f36dd154ea3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:08.729606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c26ec-58f5-460f-aa93-dd6a6215331b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:08.730421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c26ec-58f5-460f-aa93-dd6a6215331b service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:08.730800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c26ec-58f5-460f-aa93-dd6a6215331b service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:08.731257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c26ec-58f5-460f-aa93-dd6a6215331b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:08.744110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c26ec-58f5-460f-aa93-dd6a6215331b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:08.745578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c26ec-58f5-460f-aa93-dd6a6215331b service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:08.761050 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1203/2406] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:09:18.618112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:18.753012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=22 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:24.310073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420a0af-ac17-488e-a5e2-7dd84e8f62c5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:24.479238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420a0af-ac17-488e-a5e2-7dd84e8f62c5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:24.488919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:24.490132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:24.490636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:24.491152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] PATH_INFO: `/v3/users/1a42899c4ae6418b80b299b14906af6d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:24.493544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a42899c4ae6418b80b299b14906af6d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:24.497470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:24.610465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a42899c4ae6418b80b299b14906af6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:24.626514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a42899c4ae6418b80b299b14906af6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:24.660773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a42899c4ae6418b80b299b14906af6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:24.660773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a42899c4ae6418b80b299b14906af6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:24.660773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a42899c4ae6418b80b299b14906af6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:24.683718 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1204/2407] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:24 2022] DELETE /identity/v3/users/1a42899c4ae6418b80b299b14906af6d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:24.691687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:24.693146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:24.693845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:24.734309 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:24.737648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:24.829727 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1204/2408] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:24.840441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:24.842593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:24.915932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a790b99-58e7-428f-b874-579917c1312e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:24.922663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224dca2c-deaf-4d64-a4e5-8fd431de84ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:24.923693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dca2c-deaf-4d64-a4e5-8fd431de84ce service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:24.924194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dca2c-deaf-4d64-a4e5-8fd431de84ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:24.924728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dca2c-deaf-4d64-a4e5-8fd431de84ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:24.939999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224dca2c-deaf-4d64-a4e5-8fd431de84ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:24.941936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224dca2c-deaf-4d64-a4e5-8fd431de84ce service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:24.952969 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1205/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:25.371816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:25.375792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:25.410857 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99615a3-78f2-4836-ad88-205a2066f3e4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:25.428920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:25.430046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:25.430722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:25.431165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] PATH_INFO: `/v3/projects/07688894b68742acb510b3d05990b60e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:25.432913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07688894b68742acb510b3d05990b60e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:25.435625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:25.537307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07688894b68742acb510b3d05990b60e', 'request_id': 'req-6c4f606e-ddaa-4da4-b1ea-48bc99868055'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:25.566963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07688894b68742acb510b3d05990b60e', 'request_id': 'req-6c4f606e-ddaa-4da4-b1ea-48bc99868055'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:25.586810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07688894b68742acb510b3d05990b60e', 'request_id': 'req-6c4f606e-ddaa-4da4-b1ea-48bc99868055'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:25.618507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] The token cache is being invalidate because project 07688894b68742acb510b3d05990b60e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:09:25.619725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:25.621395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c4f606e-ddaa-4da4-b1ea-48bc99868055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:25.634153 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1205/2410] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:09:25 2022] DELETE /identity/v3/projects/07688894b68742acb510b3d05990b60e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:09:25.634675 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf4e4-9406-407a-9e38-9cc990998f7b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:25.635381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf4e4-9406-407a-9e38-9cc990998f7b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:25.635545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bf4e4-9406-407a-9e38-9cc990998f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:25.679371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-971bf4e4-9406-407a-9e38-9cc990998f7b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:25.682482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-971bf4e4-9406-407a-9e38-9cc990998f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:25.737748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:25.888248 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1206/2411] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:25.903293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5cf4b-8095-4334-8b5a-b88ce3a7f3e7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:25.904759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5cf4b-8095-4334-8b5a-b88ce3a7f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:25.904759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5cf4b-8095-4334-8b5a-b88ce3a7f3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:25.935566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0ff5cf4b-8095-4334-8b5a-b88ce3a7f3e7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:25.936995 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0ff5cf4b-8095-4334-8b5a-b88ce3a7f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:25.948544 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1206/2412] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:25.956770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bf4e4-9406-407a-9e38-9cc990998f7b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:25.987862 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bf4e4-9406-407a-9e38-9cc990998f7b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:25.994817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:25.995770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:25.996030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:25.996402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:25.997402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:09:25.997764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.000326 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.027465 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1207/2413] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:09:26.033969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5cf4b-8095-4334-8b5a-b88ce3a7f3e7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.052014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5cf4b-8095-4334-8b5a-b88ce3a7f3e7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.059349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.061100 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.061721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.062367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.063655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.068341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.125282 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1207/2414] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:09:26 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:26.132288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.146215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9382f3f-4e79-4be9-8e3c-e9938407f50c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.238890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.239655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.240806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.241218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.242289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.246807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.293364 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1208/2415] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:09:26 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:26.301378 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.324688 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608326b-5a31-4d92-ba37-261b13d60bba admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.348582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.352823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.353375 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.356941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.358176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.360575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.379973 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1208/2416] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:26.385067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.397893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5df8aa-9156-4067-ae9e-ccb46c31a596 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.403132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5021da-cba0-46b7-b17a-8feaac755973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.404522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5021da-cba0-46b7-b17a-8feaac755973 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.404866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5021da-cba0-46b7-b17a-8feaac755973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.405391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5021da-cba0-46b7-b17a-8feaac755973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.406250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5021da-cba0-46b7-b17a-8feaac755973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.407911 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5021da-cba0-46b7-b17a-8feaac755973 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.421570 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1209/2417] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:26.425993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.445231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710335-4ac0-4d0d-b876-a8ab67147557 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.451947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.452869 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.453183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.453690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] PATH_INFO: `/v3/projects/a238bae8dedc4995beee3e1d6805f5d6/users/45dabe2529da49158840a1bb58210f1c/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.474392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a238bae8dedc4995beee3e1d6805f5d6, user_id=45dabe2529da49158840a1bb58210f1c, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.477859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.495697 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1209/2418] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:26 2022] PUT /identity/v3/projects/a238bae8dedc4995beee3e1d6805f5d6/users/45dabe2529da49158840a1bb58210f1c/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:26.509557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35931c17-a850-4701-8ccd-ec02ba36d4c2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.509995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35931c17-a850-4701-8ccd-ec02ba36d4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.510161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35931c17-a850-4701-8ccd-ec02ba36d4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.570436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-35931c17-a850-4701-8ccd-ec02ba36d4c2 None None] MFA Rules not processed for user `45dabe2529da49158840a1bb58210f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:26.574249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-35931c17-a850-4701-8ccd-ec02ba36d4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:26.667956 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1210/2419] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:26.675138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.706692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:26.765367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b729c-f68f-4d06-91e5-1444a39544eb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.772231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.773093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.773695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.774098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.774913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.781234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.819399 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1210/2420] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:09:26 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:26.823634 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35931c17-a850-4701-8ccd-ec02ba36d4c2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.835588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35931c17-a850-4701-8ccd-ec02ba36d4c2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.853148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:26.919001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.919548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.919863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.920144 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.920859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.922949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.951310 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1211/2421] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:09:26 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:09:26.956495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:26.970034 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe56407-c367-4052-ada7-ab0533136d24 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:26.976825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:26.977902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:26.978300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:26.978667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:26.979701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:26.982152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:26.999072 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1211/2422] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:09:27.005894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.019546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd2da7-f293-44cc-9926-dc1640b4b314 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.025093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8196b71-213f-425d-878f-cca9b0c9634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.025726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8196b71-213f-425d-878f-cca9b0c9634b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.026004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8196b71-213f-425d-878f-cca9b0c9634b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.026286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8196b71-213f-425d-878f-cca9b0c9634b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.026932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8196b71-213f-425d-878f-cca9b0c9634b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.029174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8196b71-213f-425d-878f-cca9b0c9634b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.041971 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1212/2423] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:27.046723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.059060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31964139-759c-493e-8956-6e02500c46a0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.065721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.066395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.066743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.066954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] PATH_INFO: `/v3/projects/7529202bb26144d4ba168e160f6c2529/users/198577a8a7dc4a1fb2003f2a34b00857/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.086338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7529202bb26144d4ba168e160f6c2529, user_id=198577a8a7dc4a1fb2003f2a34b00857, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.092952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.111993 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1212/2424] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:27 2022] PUT /identity/v3/projects/7529202bb26144d4ba168e160f6c2529/users/198577a8a7dc4a1fb2003f2a34b00857/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:27.205064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8219-005f-4373-b16a-c94bf9f62dc6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.205376 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8219-005f-4373-b16a-c94bf9f62dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.205539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb8219-005f-4373-b16a-c94bf9f62dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.238338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-31cb8219-005f-4373-b16a-c94bf9f62dc6 None None] MFA Rules not processed for user `198577a8a7dc4a1fb2003f2a34b00857`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:27.239554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31cb8219-005f-4373-b16a-c94bf9f62dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:27.328151 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1213/2425] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:27.333278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.410383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6e6ab-97c9-46fb-8fdb-578717cbe6aa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.416543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.417222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.417542 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.417853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.418630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.421526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.459501 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1213/2426] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:09:27 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:09:27.464146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb8219-005f-4373-b16a-c94bf9f62dc6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.478863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb8219-005f-4373-b16a-c94bf9f62dc6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.583785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.584817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.585347 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.586010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.587090 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.591724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.630249 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1214/2427] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:09:27 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:27.636461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.648534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd675d-334c-45d0-a8e3-78761d681102 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.654101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.654745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.655155 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.655463 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.656302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.658220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.671723 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1214/2428] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:27.677039 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.689395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec37819-fc91-40ea-ab01-9c2cf9101a41 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.694653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.695304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.695520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.695846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] PATH_INFO: `/v3/projects/dfd964f2e3c540bfb542fb9cba238bd5/users/0bdd6168343f4ff7a4a7389982d376e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.711939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:27.716456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd964f2e3c540bfb542fb9cba238bd5, user_id=0bdd6168343f4ff7a4a7389982d376e8, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.719948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.746429 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1215/2429] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:27 2022] PUT /identity/v3/projects/dfd964f2e3c540bfb542fb9cba238bd5/users/0bdd6168343f4ff7a4a7389982d376e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:09:27.753058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.764236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857b069-5d0b-485e-9232-3a1ecc69943f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.828007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:27.859453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea3999-2f2e-4430-b86f-d32c3b174c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.860718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea3999-2f2e-4430-b86f-d32c3b174c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.861164 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea3999-2f2e-4430-b86f-d32c3b174c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.861614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea3999-2f2e-4430-b86f-d32c3b174c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.862723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea3999-2f2e-4430-b86f-d32c3b174c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.865657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea3999-2f2e-4430-b86f-d32c3b174c4a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.880256 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1215/2430] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:27.886525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:27.906393 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87c129-abed-4b76-bcaf-d38d8b636719 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:27.914900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:27.916060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.916553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.917030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] PATH_INFO: `/v3/projects/dfd964f2e3c540bfb542fb9cba238bd5/users/0bdd6168343f4ff7a4a7389982d376e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:27.920610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd964f2e3c540bfb542fb9cba238bd5, user_id=0bdd6168343f4ff7a4a7389982d376e8, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:27.923880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:27.941341 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1216/2431] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:27 2022] PUT /identity/v3/projects/dfd964f2e3c540bfb542fb9cba238bd5/users/0bdd6168343f4ff7a4a7389982d376e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:27.958958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cce2-ae74-4810-ad86-cdfb064f6aeb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:27.959628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cce2-ae74-4810-ad86-cdfb064f6aeb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:27.959628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cce2-ae74-4810-ad86-cdfb064f6aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:28.001732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7b82cce2-ae74-4810-ad86-cdfb064f6aeb None None] MFA Rules not processed for user `0bdd6168343f4ff7a4a7389982d376e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:28.002674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7b82cce2-ae74-4810-ad86-cdfb064f6aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:28.084994 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1216/2432] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:28.100757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:28.102674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:28.186243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1bbff-5b06-479e-890e-806f7557346c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:28.192600 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:28.193598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:28.194128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:28.194681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:28.278440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:28.279283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:28.279563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:28.285829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:28.288587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:28.301643 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1217/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:28.316715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:28.318866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:28.419765 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1217/2434] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:09:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:28.440784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:28.442736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:28.550574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3fd85-daef-4755-8ba9-8c7ae834658b service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:28.565348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:28.566111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:28.566613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:28.567009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:28.579512 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:28.581280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:28.595928 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1218/2435] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:09:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:09:30.362620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:30.366124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:30.404503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970279e-65a7-4618-8039-619dcedc85c1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:30.406078 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:30.407329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:30.407938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:30.408578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:30.637901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:30.639969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:30.661111 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1218/2436] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:32.026456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:32.058280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66eee6-0c05-4a7f-b7f0-c6dc42de3223 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:32.076977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2b093-a640-41c3-a7ee-7d9e45d0715b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:32.078061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2b093-a640-41c3-a7ee-7d9e45d0715b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:32.078554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2b093-a640-41c3-a7ee-7d9e45d0715b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:32.079987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2b093-a640-41c3-a7ee-7d9e45d0715b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:32.105932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2b093-a640-41c3-a7ee-7d9e45d0715b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:32.109078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2b093-a640-41c3-a7ee-7d9e45d0715b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:32.135935 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1219/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:36.773045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:36.802932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:37.030375 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea13da-12f5-486a-92d4-3c9a1811ee4c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:37.036574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:37.037468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:37.038067 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:37.038421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] PATH_INFO: `/v3/users/45dabe2529da49158840a1bb58210f1c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:37.045953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45dabe2529da49158840a1bb58210f1c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:37.050665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:37.207427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45dabe2529da49158840a1bb58210f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:37.208212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45dabe2529da49158840a1bb58210f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:37.237115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45dabe2529da49158840a1bb58210f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:37.286166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45dabe2529da49158840a1bb58210f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:37.287262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45dabe2529da49158840a1bb58210f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:37.325908 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1219/2438] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:36 2022] DELETE /identity/v3/users/45dabe2529da49158840a1bb58210f1c => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:09:37.330599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:37.331302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:37.331302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:37.367800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:37.375380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:37.383085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:37.497311 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1220/2439] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:37.508555 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:37.510225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:37.616113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c6f61-9718-4d85-b840-d4790ab29252 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:37.627694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:37.628998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:37.629711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:37.630435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:37.651338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:37.654371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:37.669658 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1220/2440] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:38.154668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:38.157136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:38.196563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59a74f-e05e-4260-9f98-738061d1e2e4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:38.216749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:38.217657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:38.218087 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:38.218519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] PATH_INFO: `/v3/projects/a238bae8dedc4995beee3e1d6805f5d6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:38.220306 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a238bae8dedc4995beee3e1d6805f5d6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:38.223056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:38.319386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a238bae8dedc4995beee3e1d6805f5d6', 'request_id': 'req-7baf4471-c122-48ba-a930-223455f82225'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.329419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a238bae8dedc4995beee3e1d6805f5d6', 'request_id': 'req-7baf4471-c122-48ba-a930-223455f82225'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.340872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a238bae8dedc4995beee3e1d6805f5d6', 'request_id': 'req-7baf4471-c122-48ba-a930-223455f82225'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.362173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] The token cache is being invalidate because project a238bae8dedc4995beee3e1d6805f5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:09:38.362822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.363722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.365955 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1221/2441] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:09:38 2022] DELETE /identity/v3/projects/a238bae8dedc4995beee3e1d6805f5d6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:38.370204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:38.383367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af1ce-3856-4eba-ab40-7717fd65af1f service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:38.447932 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:38.501066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:38.501987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:38.502488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:38.502901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] PATH_INFO: `/v3/users/198577a8a7dc4a1fb2003f2a34b00857` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:38.504294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] RBAC: Authorizing `identity:delete_user(user_id=198577a8a7dc4a1fb2003f2a34b00857)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:38.506436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:38.595251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198577a8a7dc4a1fb2003f2a34b00857'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.596108 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198577a8a7dc4a1fb2003f2a34b00857'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.605089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198577a8a7dc4a1fb2003f2a34b00857'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.622776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198577a8a7dc4a1fb2003f2a34b00857'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.623645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198577a8a7dc4a1fb2003f2a34b00857'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:38.643024 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1221/2442] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:38 2022] DELETE /identity/v3/users/198577a8a7dc4a1fb2003f2a34b00857 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:38.991538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:38.994028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:39.019072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf4471-c122-48ba-a930-223455f82225 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:39.129982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:39.131047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:39.131575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:39.131970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] PATH_INFO: `/v3/projects/7529202bb26144d4ba168e160f6c2529` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:39.134101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7529202bb26144d4ba168e160f6c2529)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:39.137601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:39.195625 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:39.211069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7529202bb26144d4ba168e160f6c2529', 'request_id': 'req-7cf879d0-1aea-4e53-82ec-618682d6a5cf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.220854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7529202bb26144d4ba168e160f6c2529', 'request_id': 'req-7cf879d0-1aea-4e53-82ec-618682d6a5cf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.230608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7529202bb26144d4ba168e160f6c2529', 'request_id': 'req-7cf879d0-1aea-4e53-82ec-618682d6a5cf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.252044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] The token cache is being invalidate because project 7529202bb26144d4ba168e160f6c2529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:09:39.252372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.253058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.254649 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1222/2443] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:09:38 2022] DELETE /identity/v3/projects/7529202bb26144d4ba168e160f6c2529 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:39.260249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:39.263686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:39.285613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed1c31-659b-4266-8967-7da9554beadf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:39.385985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:39.387970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:39.387970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:39.388367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] PATH_INFO: `/v3/users/0bdd6168343f4ff7a4a7389982d376e8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:39.390081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bdd6168343f4ff7a4a7389982d376e8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:39.393825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:39.459460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bdd6168343f4ff7a4a7389982d376e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.460515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bdd6168343f4ff7a4a7389982d376e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.468266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:39.473834 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bdd6168343f4ff7a4a7389982d376e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.499578 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bdd6168343f4ff7a4a7389982d376e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.500487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6d8916aa-c473-4b36-bd75-b354be17302f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bdd6168343f4ff7a4a7389982d376e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:39.518093 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1222/2444] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:39 2022] DELETE /identity/v3/users/0bdd6168343f4ff7a4a7389982d376e8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:40.035203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.038632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:40.063100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf879d0-1aea-4e53-82ec-618682d6a5cf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:40.141631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:40.142176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.142461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.142738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] PATH_INFO: `/v3/projects/dfd964f2e3c540bfb542fb9cba238bd5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.143951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd964f2e3c540bfb542fb9cba238bd5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:40.145947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:40.205620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd964f2e3c540bfb542fb9cba238bd5', 'request_id': 'req-58dab1ca-dd21-481e-9f68-6571df970d74'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:40.213184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd964f2e3c540bfb542fb9cba238bd5', 'request_id': 'req-58dab1ca-dd21-481e-9f68-6571df970d74'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:40.220846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd964f2e3c540bfb542fb9cba238bd5', 'request_id': 'req-58dab1ca-dd21-481e-9f68-6571df970d74'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:40.237078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] The token cache is being invalidate because project dfd964f2e3c540bfb542fb9cba238bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:09:40.237385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:40.238100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-58dab1ca-dd21-481e-9f68-6571df970d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:40.239773 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1223/2445] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:09:40 2022] DELETE /identity/v3/projects/dfd964f2e3c540bfb542fb9cba238bd5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:40.246372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788901-e705-4286-a769-a72d50426995 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.246900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788901-e705-4286-a769-a72d50426995 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.247390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788901-e705-4286-a769-a72d50426995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.274295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ec788901-e705-4286-a769-a72d50426995 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:40.275630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ec788901-e705-4286-a769-a72d50426995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:40.368566 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1223/2446] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:40.379378 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b7bf5-a6ec-4ba5-bb5d-fec300593ff9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.379664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b7bf5-a6ec-4ba5-bb5d-fec300593ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.380130 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b7bf5-a6ec-4ba5-bb5d-fec300593ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.404338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-920b7bf5-a6ec-4ba5-bb5d-fec300593ff9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:40.405514 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-920b7bf5-a6ec-4ba5-bb5d-fec300593ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:40.415066 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1224/2447] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:40.420845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788901-e705-4286-a769-a72d50426995 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.431886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788901-e705-4286-a769-a72d50426995 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:40.436979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:40.437742 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.438025 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.438312 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.438962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:09:40.439359 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:40.440939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:40.454517 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1224/2448] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:09:40.459865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b7bf5-a6ec-4ba5-bb5d-fec300593ff9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.471411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b7bf5-a6ec-4ba5-bb5d-fec300593ff9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:40.476736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:40.477359 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.477640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.477924 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.478578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:40.480544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:40.514430 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1225/2449] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:09:40 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:40.518774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.530721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154dd84-b3de-475f-9478-edd458ab4e97 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:40.597237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:40.597855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.598140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.598446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.599125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:40.601178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:40.627725 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1225/2450] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:09:40 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:40.632506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.645713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f450ba-07de-4c4c-813d-647c35e44e2d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:40.654089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:40.655162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.655587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.656021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.656777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:40.658439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:40.670579 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1226/2451] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:40.675958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.687234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81020c4d-b577-4d08-a688-7f7c8de18228 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:40.692290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94996b1-cb3c-410f-b49a-df6d64d92c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:40.692957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94996b1-cb3c-410f-b49a-df6d64d92c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.693240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94996b1-cb3c-410f-b49a-df6d64d92c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.693513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94996b1-cb3c-410f-b49a-df6d64d92c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.694160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94996b1-cb3c-410f-b49a-df6d64d92c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:40.695796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94996b1-cb3c-410f-b49a-df6d64d92c43 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:40.708508 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1226/2452] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:40.712817 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.724510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c6f5c-a1ca-416e-9637-d55fa2753814 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:40.729376 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:40.730061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.730346 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.730711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] PATH_INFO: `/v3/projects/728ff92e5bef4208bb6e1fc993515e76/users/07de4a5996ec4a6e8ad204f04e8651d1/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.745986 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:40.751323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728ff92e5bef4208bb6e1fc993515e76, user_id=07de4a5996ec4a6e8ad204f04e8651d1, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:40.755413 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:40.779554 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1227/2453] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:40 2022] PUT /identity/v3/projects/728ff92e5bef4208bb6e1fc993515e76/users/07de4a5996ec4a6e8ad204f04e8651d1/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:40.791005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:40.791373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:40.791695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:40.826436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] MFA Rules not processed for user `07de4a5996ec4a6e8ad204f04e8651d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:40.827684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:40.857200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:40.909505 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1227/2454] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:40.922864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:40.924737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:41.002414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9d229-de06-4b05-9858-dd197f028c99 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:41.006387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1195cc-29fa-4758-be04-70f7660857a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:41.007026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1195cc-29fa-4758-be04-70f7660857a9 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:41.007304 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1195cc-29fa-4758-be04-70f7660857a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:41.007576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1195cc-29fa-4758-be04-70f7660857a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:41.020032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1195cc-29fa-4758-be04-70f7660857a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:41.022110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1195cc-29fa-4758-be04-70f7660857a9 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:41.039338 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1228/2455] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:50.906031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=16 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:50.978940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:50.981255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:51.031231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:51.057357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eceb3c-fa17-4656-9d75-0758f2a4d593 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:51.061306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:51.061941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:51.062221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:51.062644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:51.064520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:51.143331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:51.145326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:51.158977 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1228/2456] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:53.223658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:53.224196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:53.224554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:53.255375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:53.256717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:53.361520 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1229/2457] 10.222.0.26 () {58 vars in 967 bytes} [Sun Sep 25 01:09:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:53.379841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:53.381484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:53.493883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79c525-bca9-4f7e-b770-dc2e4bd1cff0 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:53.501416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:53.502358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:53.503126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:53.503695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:53.519877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:53.521623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:53.532110 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1229/2458] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:09:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:09:56.474253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:56.475733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:56.490942 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ec39b-3f97-471c-8809-418d9dbbc570 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:56.495610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:56.496283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:56.496602 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:56.496898 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] PATH_INFO: `/v3/users/f32c9b07e6234d55b0d4846d44ee6ce6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:56.498078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32c9b07e6234d55b0d4846d44ee6ce6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:56.500139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:56.564421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32c9b07e6234d55b0d4846d44ee6ce6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.575948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32c9b07e6234d55b0d4846d44ee6ce6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.591811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32c9b07e6234d55b0d4846d44ee6ce6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.592078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32c9b07e6234d55b0d4846d44ee6ce6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.592699 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d7e1305-604b-4a06-8a2f-62d830a94133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32c9b07e6234d55b0d4846d44ee6ce6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.607221 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1230/2459] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:56 2022] DELETE /identity/v3/users/f32c9b07e6234d55b0d4846d44ee6ce6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:56.780130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:56.792392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d871d-638d-4ce6-8547-6977c3628e64 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:56.851223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:56.874140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:56.874779 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:56.875065 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:56.875341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] PATH_INFO: `/v3/projects/537d0a114f314e9b948150001bda642c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:56.876595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=537d0a114f314e9b948150001bda642c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:56.878459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:56.930856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537d0a114f314e9b948150001bda642c', 'request_id': 'req-2edcad2f-1007-4768-aa02-2f8c60f23ee0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.941691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537d0a114f314e9b948150001bda642c', 'request_id': 'req-2edcad2f-1007-4768-aa02-2f8c60f23ee0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.948228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537d0a114f314e9b948150001bda642c', 'request_id': 'req-2edcad2f-1007-4768-aa02-2f8c60f23ee0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.960416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] The token cache is being invalidate because project 537d0a114f314e9b948150001bda642c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:09:56.960720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.961391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2edcad2f-1007-4768-aa02-2f8c60f23ee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:56.963036 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1230/2460] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:09:56 2022] DELETE /identity/v3/projects/537d0a114f314e9b948150001bda642c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:56.968517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4f321-5391-473a-b096-e30ac19155c0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:56.969010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4f321-5391-473a-b096-e30ac19155c0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:56.969497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4f321-5391-473a-b096-e30ac19155c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:56.993578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-24f4f321-5391-473a-b096-e30ac19155c0 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:56.995032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-24f4f321-5391-473a-b096-e30ac19155c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:57.070904 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1231/2461] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:57.080063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c78e-cdac-4f81-9bcc-8391fc7ef4c9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.080308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c78e-cdac-4f81-9bcc-8391fc7ef4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.080614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c78e-cdac-4f81-9bcc-8391fc7ef4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.101059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dcc8c78e-cdac-4f81-9bcc-8391fc7ef4c9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:57.102345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dcc8c78e-cdac-4f81-9bcc-8391fc7ef4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:57.112920 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1231/2462] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:57.117142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4f321-5391-473a-b096-e30ac19155c0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.127613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4f321-5391-473a-b096-e30ac19155c0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.132772 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.133387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.133670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.133941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.134607 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:09:57.134906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.137371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.150081 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1232/2463] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:09:57.154453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8c78e-cdac-4f81-9bcc-8391fc7ef4c9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.163953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8c78e-cdac-4f81-9bcc-8391fc7ef4c9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.169319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.170012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.170353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.170709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.171414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.173383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.209903 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1232/2464] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:09:57 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:57.214576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.221927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:57.226223 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8052-93fc-4a57-b8bd-af3a04564675 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.315687 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.317113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.317865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.318464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.319203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.322051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.355640 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1233/2465] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:09:57 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:09:57.359948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.372250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14ef90-ad5e-40fc-a045-ca17c96b188d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.380042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.380702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.380984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.381331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.381981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.383616 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.396132 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1233/2466] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:57.400197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.411708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6c733-22db-4d6d-bf18-b99119fc39c9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.417808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2841ef-54a5-44ac-bc9e-9c6dddfdedd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.418433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2841ef-54a5-44ac-bc9e-9c6dddfdedd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.418705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2841ef-54a5-44ac-bc9e-9c6dddfdedd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.419059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2841ef-54a5-44ac-bc9e-9c6dddfdedd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.419818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2841ef-54a5-44ac-bc9e-9c6dddfdedd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.421476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2841ef-54a5-44ac-bc9e-9c6dddfdedd7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.434403 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1234/2467] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:57.438984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.449631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d339b-2582-4793-813e-dc37adeae7c9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.454954 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.456367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.456737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.457010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] PATH_INFO: `/v3/projects/22cc4210c8f64262b332c4eb31a6e1bf/users/fa82b8c654424193afda5d5f00a38d67/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.475114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22cc4210c8f64262b332c4eb31a6e1bf, user_id=fa82b8c654424193afda5d5f00a38d67, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.477849 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.497340 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1234/2468] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:57 2022] PUT /identity/v3/projects/22cc4210c8f64262b332c4eb31a6e1bf/users/fa82b8c654424193afda5d5f00a38d67/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:57.508593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadda7c6-81c8-4101-817e-f9d2f98d11e7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.509186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadda7c6-81c8-4101-817e-f9d2f98d11e7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.509186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadda7c6-81c8-4101-817e-f9d2f98d11e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.545863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cadda7c6-81c8-4101-817e-f9d2f98d11e7 None None] MFA Rules not processed for user `fa82b8c654424193afda5d5f00a38d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:57.547207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cadda7c6-81c8-4101-817e-f9d2f98d11e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:57.634737 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1235/2469] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:57.639809 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.720715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bf5bf-210a-4df2-a9c5-4f91b84bad40 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.727966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.728633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.728965 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.730042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.730696 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.732899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.749471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:57.770078 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1235/2470] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:09:57 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:57.774437 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadda7c6-81c8-4101-817e-f9d2f98d11e7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.785736 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadda7c6-81c8-4101-817e-f9d2f98d11e7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.859849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.860614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.860992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.861360 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.862352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.864097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.890937 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1236/2471] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:09:57 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:09:57.895338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.906380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f8262-346d-4365-afe1-4fb39b389668 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.912080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.913172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.913349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.913546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.914184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.915775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.926887 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1236/2472] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:09:57.931364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:57.937331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:57.942842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51494788-e85f-4cae-8327-92cde97e2e3a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:57.949965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:57.951016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:57.951735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:57.952022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] PATH_INFO: `/v3/projects/38152565dca041ee8a8c45ac6ecaede4/users/32e47429bf384521aef533202dc12ebf/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:57.970969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38152565dca041ee8a8c45ac6ecaede4, user_id=32e47429bf384521aef533202dc12ebf, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:57.974103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:57.992654 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1237/2473] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:57 2022] PUT /identity/v3/projects/38152565dca041ee8a8c45ac6ecaede4/users/32e47429bf384521aef533202dc12ebf/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:57.997221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:58.008080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9af679-b0a4-451d-9cab-e584ea70ea5a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:58.070001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd85b9-6fda-49cd-b388-aedcf6e58a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:58.071502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd85b9-6fda-49cd-b388-aedcf6e58a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:58.071502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd85b9-6fda-49cd-b388-aedcf6e58a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:58.071502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd85b9-6fda-49cd-b388-aedcf6e58a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:58.072026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd85b9-6fda-49cd-b388-aedcf6e58a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:58.073736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd85b9-6fda-49cd-b388-aedcf6e58a0e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:58.090160 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1237/2474] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:09:58.091952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:58.102937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb199f-dda2-4080-bb2f-2d6b2c570f10 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:58.108834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:58.109498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:58.109698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:58.109978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] PATH_INFO: `/v3/projects/38152565dca041ee8a8c45ac6ecaede4/users/32e47429bf384521aef533202dc12ebf/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:58.111959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38152565dca041ee8a8c45ac6ecaede4, user_id=32e47429bf384521aef533202dc12ebf, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:58.115032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:58.130009 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1238/2475] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:09:58 2022] PUT /identity/v3/projects/38152565dca041ee8a8c45ac6ecaede4/users/32e47429bf384521aef533202dc12ebf/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:09:58.141213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa861b1-8bde-4e4a-bbda-d6562e191e72 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:58.141635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa861b1-8bde-4e4a-bbda-d6562e191e72 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:58.141986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa861b1-8bde-4e4a-bbda-d6562e191e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:58.173040 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9fa861b1-8bde-4e4a-bbda-d6562e191e72 None None] MFA Rules not processed for user `32e47429bf384521aef533202dc12ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:58.174261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9fa861b1-8bde-4e4a-bbda-d6562e191e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:58.255334 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1238/2476] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:09:58.268634 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:58.270485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:58.336118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9e20d-8777-4b29-a2b7-38536f6f1eb2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:58.341691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e24f31-c3dd-4e23-ae7c-85a7935d11ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:58.341691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e24f31-c3dd-4e23-ae7c-85a7935d11ae service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:58.342026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e24f31-c3dd-4e23-ae7c-85a7935d11ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:58.342142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e24f31-c3dd-4e23-ae7c-85a7935d11ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:58.351833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e24f31-c3dd-4e23-ae7c-85a7935d11ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:58.353328 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e24f31-c3dd-4e23-ae7c-85a7935d11ae service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:58.364537 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1239/2477] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:58.793338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa861b1-8bde-4e4a-bbda-d6562e191e72 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:58.872727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa861b1-8bde-4e4a-bbda-d6562e191e72 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:58.878999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:58.879660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:58.879939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:58.880209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] PATH_INFO: `/v3/users/fa82b8c654424193afda5d5f00a38d67` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:58.881437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa82b8c654424193afda5d5f00a38d67)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:58.883395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:58.886369 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:58.961102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa82b8c654424193afda5d5f00a38d67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:58.961895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa82b8c654424193afda5d5f00a38d67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:58.970406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa82b8c654424193afda5d5f00a38d67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:58.988208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa82b8c654424193afda5d5f00a38d67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:58.988931 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa82b8c654424193afda5d5f00a38d67'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.005528 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1239/2478] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:58 2022] DELETE /identity/v3/users/fa82b8c654424193afda5d5f00a38d67 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:59.010553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73eb89-dd11-4294-84a8-0b6c88189a54 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:59.011201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73eb89-dd11-4294-84a8-0b6c88189a54 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:59.011201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73eb89-dd11-4294-84a8-0b6c88189a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:59.031652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3e73eb89-dd11-4294-84a8-0b6c88189a54 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:59.033071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3e73eb89-dd11-4294-84a8-0b6c88189a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:59.118093 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1240/2479] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:59.128847 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:59.131206 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:59.148837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba62b5-7152-4ee8-a2d5-a9548e1522a8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:59.149214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba62b5-7152-4ee8-a2d5-a9548e1522a8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:59.149489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba62b5-7152-4ee8-a2d5-a9548e1522a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:59.172770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2cba62b5-7152-4ee8-a2d5-a9548e1522a8 None None] MFA Rules not processed for user `9d73ba36238f449dbe08d8888920510f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:09:59.174001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2cba62b5-7152-4ee8-a2d5-a9548e1522a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:59.205462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea78bd-aa3d-4efb-bddb-4ba8492cab82 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:59.209823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:59.210445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:59.210723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:59.210992 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:59.221515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:59.222967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:59.234914 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1240/2480] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:09:59.240296 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1241/2481] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:09:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:09:59.562321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:59.563905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:09:59.581753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02484a8-bedf-4496-9369-1ddc9c01c09e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:59.587972 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:59.588946 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:59.589797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:59.590138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] PATH_INFO: `/v3/projects/22cc4210c8f64262b332c4eb31a6e1bf` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:59.591449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22cc4210c8f64262b332c4eb31a6e1bf)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:59.593536 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:59.643347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22cc4210c8f64262b332c4eb31a6e1bf', 'request_id': 'req-018499ef-52fd-4f13-9739-b083fed557b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.652177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22cc4210c8f64262b332c4eb31a6e1bf', 'request_id': 'req-018499ef-52fd-4f13-9739-b083fed557b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.663287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22cc4210c8f64262b332c4eb31a6e1bf', 'request_id': 'req-018499ef-52fd-4f13-9739-b083fed557b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.674800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] The token cache is being invalidate because project 22cc4210c8f64262b332c4eb31a6e1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:09:59.675083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.675749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.677260 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1241/2482] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:09:59 2022] DELETE /identity/v3/projects/22cc4210c8f64262b332c4eb31a6e1bf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:09:59.681380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba62b5-7152-4ee8-a2d5-a9548e1522a8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:09:59.692308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba62b5-7152-4ee8-a2d5-a9548e1522a8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:09:59.702286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:59.757073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:09:59.757778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:09:59.758063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:09:59.758375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] PATH_INFO: `/v3/users/32e47429bf384521aef533202dc12ebf` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:09:59.759663 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e47429bf384521aef533202dc12ebf)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:09:59.762295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:09:59.818527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e47429bf384521aef533202dc12ebf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.826945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e47429bf384521aef533202dc12ebf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.841807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e47429bf384521aef533202dc12ebf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.842544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e47429bf384521aef533202dc12ebf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.843286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f6d73f4-7e6d-4a96-b4fd-49de3cfdf3d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e47429bf384521aef533202dc12ebf'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:09:59.857285 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1242/2483] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:09:59 2022] DELETE /identity/v3/users/32e47429bf384521aef533202dc12ebf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:00.217996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:00.221138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:00.254335 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018499ef-52fd-4f13-9739-b083fed557b4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:00.259451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:00.327999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:00.328913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.329212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.329486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] PATH_INFO: `/v3/projects/38152565dca041ee8a8c45ac6ecaede4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.330748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=38152565dca041ee8a8c45ac6ecaede4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:00.332985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:00.389902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38152565dca041ee8a8c45ac6ecaede4', 'request_id': 'req-2e866756-c900-43c3-ab15-d855d7c05c0f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:00.399905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38152565dca041ee8a8c45ac6ecaede4', 'request_id': 'req-2e866756-c900-43c3-ab15-d855d7c05c0f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:00.406187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38152565dca041ee8a8c45ac6ecaede4', 'request_id': 'req-2e866756-c900-43c3-ab15-d855d7c05c0f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:00.419245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] The token cache is being invalidate because project 38152565dca041ee8a8c45ac6ecaede4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:00.419553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:00.420211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2e866756-c900-43c3-ab15-d855d7c05c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:00.421737 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1242/2484] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:00 2022] DELETE /identity/v3/projects/38152565dca041ee8a8c45ac6ecaede4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:00.426730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9366a-f302-4fd9-b65b-8a5e0d2a758f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.427486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9366a-f302-4fd9-b65b-8a5e0d2a758f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.427486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9366a-f302-4fd9-b65b-8a5e0d2a758f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.448035 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0ed9366a-f302-4fd9-b65b-8a5e0d2a758f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:00.449925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0ed9366a-f302-4fd9-b65b-8a5e0d2a758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:00.525573 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1243/2485] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:10:00.536844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49f3fa-ea08-4d01-a987-3974a9853162 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.537054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49f3fa-ea08-4d01-a987-3974a9853162 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.537792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49f3fa-ea08-4d01-a987-3974a9853162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.560122 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9f49f3fa-ea08-4d01-a987-3974a9853162 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:00.561553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9f49f3fa-ea08-4d01-a987-3974a9853162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:00.570813 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1243/2486] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:00.576513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9366a-f302-4fd9-b65b-8a5e0d2a758f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:00.587001 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9366a-f302-4fd9-b65b-8a5e0d2a758f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:00.593686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:00.594586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.594968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.595373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.596042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:00.596371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:00.598206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:00.612424 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1244/2487] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:10:00.616331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49f3fa-ea08-4d01-a987-3974a9853162 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:00.627442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49f3fa-ea08-4d01-a987-3974a9853162 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:00.633162 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:00.634033 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.634553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.635015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.635687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:00.638249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:00.673754 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1244/2488] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:00 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:10:00.674965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:00.687079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3d0f6-0832-4142-92f7-2f80539c1de6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:00.705640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:00.751345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:00.751967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.752246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.752548 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.753213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:00.755255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:00.777421 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1245/2489] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:00 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:00.781608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:00.792583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd452-0701-4c7f-a4ed-0127c298a972 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:00.797428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:00.798089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.798426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.798703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.799346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:00.800986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:00.812622 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1245/2490] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:10:00.816795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:00.828014 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cf3c2-d1c5-4055-8ccf-5eac9b8f0fd1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:00.832963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a20f5-9174-47b4-9651-a1a0e0edc4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:00.833571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a20f5-9174-47b4-9651-a1a0e0edc4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.833855 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a20f5-9174-47b4-9651-a1a0e0edc4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.834176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a20f5-9174-47b4-9651-a1a0e0edc4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.834903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a20f5-9174-47b4-9651-a1a0e0edc4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:00.836614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a20f5-9174-47b4-9651-a1a0e0edc4f1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:00.848344 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1246/2491] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:00.852660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:00.862756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969dc0-d06a-4466-b9b2-8787aea52ebc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:00.867681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:00.868338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.868653 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.868930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] PATH_INFO: `/v3/projects/c183c3af3f8e4a80b4ca6510b48d552d/users/9b2820acddc342c3820cf4d92fb81db8/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.884494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c183c3af3f8e4a80b4ca6510b48d552d, user_id=9b2820acddc342c3820cf4d92fb81db8, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:00.888066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:00.901898 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1246/2492] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:00 2022] PUT /identity/v3/projects/c183c3af3f8e4a80b4ca6510b48d552d/users/9b2820acddc342c3820cf4d92fb81db8/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:00.912803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e7c7-1cc5-4a09-bb10-7a8c28fb1826 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:00.913088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e7c7-1cc5-4a09-bb10-7a8c28fb1826 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:00.913364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205e7c7-1cc5-4a09-bb10-7a8c28fb1826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:00.944276 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0205e7c7-1cc5-4a09-bb10-7a8c28fb1826 None None] MFA Rules not processed for user `9b2820acddc342c3820cf4d92fb81db8`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:00.945533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0205e7c7-1cc5-4a09-bb10-7a8c28fb1826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:01.025201 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1247/2493] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:01.030466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:01.080919 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:01.127880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01d553-8f2d-4bdf-a22c-31af8a4e7d1c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:01.149124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:01.149124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.149124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.149124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.149554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:01.149554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:01.184348 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1247/2494] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:01 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:10:01.188476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205e7c7-1cc5-4a09-bb10-7a8c28fb1826 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:01.204783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205e7c7-1cc5-4a09-bb10-7a8c28fb1826 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:01.291744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:01.293105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.293387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.293654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.294396 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:01.297738 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:01.342771 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1248/2495] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:01 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:10:01.350533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:01.363792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6633e-11a0-4f86-b8f1-0ac906a5e7cd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:01.369462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:01.370197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.370487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.370788 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.371492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:01.373278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:01.387346 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1248/2496] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:01.392680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:01.411121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7f2dd-e76f-4408-af8d-aff44f4df4f5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:01.420913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:01.421591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.421922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.422201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] PATH_INFO: `/v3/projects/7c0c64040c2e45388be1b40740c9dbb7/users/f63ef329b72d4c58aab9cdc333152cb6/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.440652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0c64040c2e45388be1b40740c9dbb7, user_id=f63ef329b72d4c58aab9cdc333152cb6, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:01.444445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:01.447395 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:01.469288 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1249/2497] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:01 2022] PUT /identity/v3/projects/7c0c64040c2e45388be1b40740c9dbb7/users/f63ef329b72d4c58aab9cdc333152cb6/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:01.471459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:01.491623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e462f5-0e32-44c4-921a-3344acc5e300 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:01.583243 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67966dc6-fd38-4af0-b152-2f3de29cdb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:01.583910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67966dc6-fd38-4af0-b152-2f3de29cdb8b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.584202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67966dc6-fd38-4af0-b152-2f3de29cdb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.584602 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67966dc6-fd38-4af0-b152-2f3de29cdb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.585184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67966dc6-fd38-4af0-b152-2f3de29cdb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:01.586908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67966dc6-fd38-4af0-b152-2f3de29cdb8b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:01.602768 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1249/2498] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:01.609074 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:01.626798 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41389018-e97b-40e9-99cb-d61c2e08dab8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:01.632374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:01.633224 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.633541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.634059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] PATH_INFO: `/v3/projects/7c0c64040c2e45388be1b40740c9dbb7/users/f63ef329b72d4c58aab9cdc333152cb6/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.637807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0c64040c2e45388be1b40740c9dbb7, user_id=f63ef329b72d4c58aab9cdc333152cb6, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:01.641480 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:01.654412 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1250/2499] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:01 2022] PUT /identity/v3/projects/7c0c64040c2e45388be1b40740c9dbb7/users/f63ef329b72d4c58aab9cdc333152cb6/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:01.671638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.671971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.672259 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.716219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] MFA Rules not processed for user `f63ef329b72d4c58aab9cdc333152cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:01.717001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:01.853353 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1250/2500] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:01.866409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:01.869252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:01.970740 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dde1e0-29c0-4b64-adb2-62cb45e199fe admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:01.975871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:01.976958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:01.977498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:01.977786 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:01.990595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:01.991975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:02.003563 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1251/2501] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:02.056662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:02.060300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:02.084856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cce4c4-80e6-4b4c-9fb7-822ad0842373 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:02.094630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5de3e-8705-433a-9dfa-9bd36866bf2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:02.095314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5de3e-8705-433a-9dfa-9bd36866bf2a service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:02.095641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5de3e-8705-433a-9dfa-9bd36866bf2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:02.096007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5de3e-8705-433a-9dfa-9bd36866bf2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:02.134080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:02.220998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5de3e-8705-433a-9dfa-9bd36866bf2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:02.226211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5de3e-8705-433a-9dfa-9bd36866bf2a service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:02.261899 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1251/2502] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:10:05.892610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:05.974332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b283610-25cc-4c74-be4d-11fb3b355825 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:05.985280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:05.986059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:05.986583 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:05.986983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] PATH_INFO: `/v3/users/9b2820acddc342c3820cf4d92fb81db8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:05.989369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2820acddc342c3820cf4d92fb81db8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:05.991522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:06.058069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2820acddc342c3820cf4d92fb81db8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.068102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2820acddc342c3820cf4d92fb81db8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.091094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2820acddc342c3820cf4d92fb81db8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.091857 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2820acddc342c3820cf4d92fb81db8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.092572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2820acddc342c3820cf4d92fb81db8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.106241 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1252/2503] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:05 2022] DELETE /identity/v3/users/9b2820acddc342c3820cf4d92fb81db8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:06.111066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:06.111603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:06.111603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:06.132478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:06.133822 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:06.200246 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1252/2504] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:06.210670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:06.212215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:06.220433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:06.284992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90729d-4c4d-4f4d-b19d-97cbb01e7bd6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:06.290419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:06.291115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:06.291402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:06.291760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:06.302406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:06.303872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:06.320189 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1253/2505] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:06.661669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:06.663482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:06.689911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1987fd4-51d7-4929-ab4b-0137ec0af722 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:06.696265 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:06.697331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:06.697915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:06.699095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] PATH_INFO: `/v3/projects/c183c3af3f8e4a80b4ca6510b48d552d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:06.700512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c183c3af3f8e4a80b4ca6510b48d552d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:06.702691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:06.785520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c183c3af3f8e4a80b4ca6510b48d552d', 'request_id': 'req-ca45b318-119c-4811-b9bc-05bf1e712ec7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.797431 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c183c3af3f8e4a80b4ca6510b48d552d', 'request_id': 'req-ca45b318-119c-4811-b9bc-05bf1e712ec7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.808047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c183c3af3f8e4a80b4ca6510b48d552d', 'request_id': 'req-ca45b318-119c-4811-b9bc-05bf1e712ec7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.824559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] The token cache is being invalidate because project c183c3af3f8e4a80b4ca6510b48d552d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:06.825332 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.825824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.827801 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1253/2506] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:06 2022] DELETE /identity/v3/projects/c183c3af3f8e4a80b4ca6510b48d552d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:06.832266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:06.843716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24191-0499-4648-9513-40db21ddddc6 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:06.923867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:06.925041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:06.925432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:06.925808 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] PATH_INFO: `/v3/users/f63ef329b72d4c58aab9cdc333152cb6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:06.927122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f63ef329b72d4c58aab9cdc333152cb6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:06.929431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:06.985659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f63ef329b72d4c58aab9cdc333152cb6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:06.994275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f63ef329b72d4c58aab9cdc333152cb6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.006998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f63ef329b72d4c58aab9cdc333152cb6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.007658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f63ef329b72d4c58aab9cdc333152cb6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.008255 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83301787-3415-479f-b77d-1cbc4e393845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f63ef329b72d4c58aab9cdc333152cb6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.021091 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1254/2507] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:06 2022] DELETE /identity/v3/users/f63ef329b72d4c58aab9cdc333152cb6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:07.366448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:07.368634 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:07.384629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45b318-119c-4811-b9bc-05bf1e712ec7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:07.443991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:07.456891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:07.457420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.457693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.457964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] PATH_INFO: `/v3/projects/7c0c64040c2e45388be1b40740c9dbb7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.459291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0c64040c2e45388be1b40740c9dbb7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:07.461384 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:07.511029 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0c64040c2e45388be1b40740c9dbb7', 'request_id': 'req-abd31636-541b-4923-8c52-dae4a9fb72e7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.520248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0c64040c2e45388be1b40740c9dbb7', 'request_id': 'req-abd31636-541b-4923-8c52-dae4a9fb72e7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.527483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0c64040c2e45388be1b40740c9dbb7', 'request_id': 'req-abd31636-541b-4923-8c52-dae4a9fb72e7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.539264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] The token cache is being invalidate because project 7c0c64040c2e45388be1b40740c9dbb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:07.539550 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.540200 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd31636-541b-4923-8c52-dae4a9fb72e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:07.542495 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1254/2508] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:07 2022] DELETE /identity/v3/projects/7c0c64040c2e45388be1b40740c9dbb7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:07.547604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371098-dde9-4824-a15d-d3566737c0ed None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.547807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371098-dde9-4824-a15d-d3566737c0ed None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.548122 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67371098-dde9-4824-a15d-d3566737c0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.569866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-67371098-dde9-4824-a15d-d3566737c0ed None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:07.571173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-67371098-dde9-4824-a15d-d3566737c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:07.592852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:07.640267 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1255/2509] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:07.650009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6af6ad-1db3-4d6b-91fe-b98586e997ad None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.650299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6af6ad-1db3-4d6b-91fe-b98586e997ad None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.650576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6af6ad-1db3-4d6b-91fe-b98586e997ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.673249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2b6af6ad-1db3-4d6b-91fe-b98586e997ad None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:07.674670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b6af6ad-1db3-4d6b-91fe-b98586e997ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:07.682779 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1255/2510] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:07.687311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67371098-dde9-4824-a15d-d3566737c0ed None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:07.698431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67371098-dde9-4824-a15d-d3566737c0ed None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:07.703636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:07.704284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.704722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.704941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.705623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:07.705887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:07.707508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:07.720913 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1256/2511] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:10:07.725694 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6af6ad-1db3-4d6b-91fe-b98586e997ad None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:07.737170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6af6ad-1db3-4d6b-91fe-b98586e997ad None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:07.742601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:07.743275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.743611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.743933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.744657 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:07.746727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:07.775454 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1256/2512] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:07 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:07.780314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:07.791267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412137c2-af8d-45d7-8b15-4f00fbae6c96 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:07.850386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:07.851107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.851282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.851563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.852190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:07.854288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:07.878867 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1257/2513] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:07 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:07.883125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:07.897196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed440-1ffd-4be1-a4a2-9f93aa569eb0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:07.904099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:07.904763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.905044 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.905345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.906004 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:07.907630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:07.928415 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1257/2514] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:07.930877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:07.943867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f6a94-95b2-4290-80dd-0fdec9da642c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:07.950532 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c515714-c037-4b3d-af7f-acf61aa690f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:07.951290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c515714-c037-4b3d-af7f-acf61aa690f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:07.951662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c515714-c037-4b3d-af7f-acf61aa690f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:07.951973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c515714-c037-4b3d-af7f-acf61aa690f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:07.953079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c515714-c037-4b3d-af7f-acf61aa690f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:07.955349 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c515714-c037-4b3d-af7f-acf61aa690f6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:07.973005 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1258/2515] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:07.974456 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:07.992478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344176b-d8ca-45f8-b2c2-8ee3ae5d1029 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.001603 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.002502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.003033 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.003319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] PATH_INFO: `/v3/projects/b9487c7d7abe48fab3402ce97dc5c66f/users/4e339af8d23a4634a2261b6430db3b37/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.027797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9487c7d7abe48fab3402ce97dc5c66f, user_id=4e339af8d23a4634a2261b6430db3b37, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.032166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.047752 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1258/2516] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:07 2022] PUT /identity/v3/projects/b9487c7d7abe48fab3402ce97dc5c66f/users/4e339af8d23a4634a2261b6430db3b37/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:08.059294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754042-35df-4d4c-b35e-77e74fe9942c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.059640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754042-35df-4d4c-b35e-77e74fe9942c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.059945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754042-35df-4d4c-b35e-77e74fe9942c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.091671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb754042-35df-4d4c-b35e-77e74fe9942c None None] MFA Rules not processed for user `4e339af8d23a4634a2261b6430db3b37`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:08.093053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb754042-35df-4d4c-b35e-77e74fe9942c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:08.168218 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1259/2517] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:08.173013 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.245797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d949a-3e1c-4b06-aa11-c6995ee40d2b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.251725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.252507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.252790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.253191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.254363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.256638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.290108 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1259/2518] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:08 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:08.294875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb754042-35df-4d4c-b35e-77e74fe9942c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.306671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb754042-35df-4d4c-b35e-77e74fe9942c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.332055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:08.390163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.390864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.391076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.391336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.392095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.394228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.427125 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1260/2519] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:08 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:10:08.428207 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.434581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:08.441769 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e5d75-d172-4b29-b564-824960304b4c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.449753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.450999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.451452 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.451973 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.453067 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.455498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.468692 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1260/2520] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:10:08.473405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.487488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397d12b-2872-4da1-a65e-6e4ccf88a0b4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.494578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d6a51-3053-4efb-aa41-e9c47f0eb648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.496419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d6a51-3053-4efb-aa41-e9c47f0eb648 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.496866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d6a51-3053-4efb-aa41-e9c47f0eb648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.497287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d6a51-3053-4efb-aa41-e9c47f0eb648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.498153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d6a51-3053-4efb-aa41-e9c47f0eb648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.500686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d6a51-3053-4efb-aa41-e9c47f0eb648 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.513159 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1261/2521] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:08.517917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.529652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb14bae-28f9-4511-9d79-fbbf2d10ccf7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.535015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.535672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.535952 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.536226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] PATH_INFO: `/v3/projects/bfc70072538444f9ad61ae15b984f6f1/users/add78d78a2af45e0b47eb43ba02389ac/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.553097 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc70072538444f9ad61ae15b984f6f1, user_id=add78d78a2af45e0b47eb43ba02389ac, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.555762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.570601 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1261/2522] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:08 2022] PUT /identity/v3/projects/bfc70072538444f9ad61ae15b984f6f1/users/add78d78a2af45e0b47eb43ba02389ac/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:08.581915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8542a7-dfff-4c1e-a9ce-91fdc9b14f0d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.582201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8542a7-dfff-4c1e-a9ce-91fdc9b14f0d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.582479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8542a7-dfff-4c1e-a9ce-91fdc9b14f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.616105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-be8542a7-dfff-4c1e-a9ce-91fdc9b14f0d None None] MFA Rules not processed for user `add78d78a2af45e0b47eb43ba02389ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:08.617364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-be8542a7-dfff-4c1e-a9ce-91fdc9b14f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:08.691412 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1262/2523] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:08.696113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.763597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923105a9-28de-4dc0-81c7-17b3b01594d3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.768685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.769302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.769582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.769923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.770517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.772726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.812059 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1262/2524] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:08 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:08.816277 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8542a7-dfff-4c1e-a9ce-91fdc9b14f0d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.829499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8542a7-dfff-4c1e-a9ce-91fdc9b14f0d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.895131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.895743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.896010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.896328 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.896982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.899037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.922551 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1263/2525] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:08 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:08.927217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.939148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49871705-090a-42ec-990b-1ef3707baf43 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.944255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.944891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.945185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.945478 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:08.946193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:08.947895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:08.960680 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1263/2526] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:08.965994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:08.977146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ba239-559a-4468-84fa-cd5022ef687c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:08.982990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:08.983834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:08.984206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:08.984731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] PATH_INFO: `/v3/projects/c96dc00c9bc146fda6c1b357f45757c0/users/ad7e019f7a8941ac8b6b3b4a929065c4/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.004542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96dc00c9bc146fda6c1b357f45757c0, user_id=ad7e019f7a8941ac8b6b3b4a929065c4, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:09.007250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:09.009999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:09.012191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:09.014086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.025117 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1264/2527] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:08 2022] PUT /identity/v3/projects/c96dc00c9bc146fda6c1b357f45757c0/users/ad7e019f7a8941ac8b6b3b4a929065c4/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:09.029717 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:09.030401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87242327-8ad4-435d-9ec4-8c3402de746e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:09.098096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:09.098872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.099151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.099436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] PATH_INFO: `/v3/users/07de4a5996ec4a6e8ad204f04e8651d1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.100645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07de4a5996ec4a6e8ad204f04e8651d1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:09.102837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:09.103749 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764b2cf-b387-43d5-831b-7380ac4e69e3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:09.110392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:09.111853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.112145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.112524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.113181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:09.114814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:09.126326 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1265/2528] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:09.130510 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:09.141357 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e419cf-f86b-4ffc-8470-4d9bf1e83b73 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:09.148265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db706be-0dd8-4b3c-b505-04493f6067e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:09.149146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db706be-0dd8-4b3c-b505-04493f6067e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.149520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db706be-0dd8-4b3c-b505-04493f6067e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.149983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db706be-0dd8-4b3c-b505-04493f6067e2 admin admin] PATH_INFO: `/v3/projects/c96dc00c9bc146fda6c1b357f45757c0/users/ad7e019f7a8941ac8b6b3b4a929065c4/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.153113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db706be-0dd8-4b3c-b505-04493f6067e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96dc00c9bc146fda6c1b357f45757c0, user_id=ad7e019f7a8941ac8b6b3b4a929065c4, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:09.155971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db706be-0dd8-4b3c-b505-04493f6067e2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:09.158341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:09.161429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07de4a5996ec4a6e8ad204f04e8651d1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.162708 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07de4a5996ec4a6e8ad204f04e8651d1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.169089 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1266/2529] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:09 2022] PUT /identity/v3/projects/c96dc00c9bc146fda6c1b357f45757c0/users/ad7e019f7a8941ac8b6b3b4a929065c4/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:09.172684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07de4a5996ec4a6e8ad204f04e8651d1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.179654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.179971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.180217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.185443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07de4a5996ec4a6e8ad204f04e8651d1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.186177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed5379cd-c27c-4928-aebe-78ba6208d876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07de4a5996ec4a6e8ad204f04e8651d1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.200801 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1264/2530] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:09 2022] DELETE /identity/v3/users/07de4a5996ec4a6e8ad204f04e8651d1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:09.205430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.205735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.206047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.211319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] MFA Rules not processed for user `ad7e019f7a8941ac8b6b3b4a929065c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:09.212578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.226398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:09.227604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.288531 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1267/2531] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:09.296305 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1265/2532] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:09.300739 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:09.302180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.310133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:09.311792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.370977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5f1e4-e32b-410e-adb1-a47100c17bde None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:09.374905 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:09.375513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.375790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.376021 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb3f1c-ea9a-457c-8efc-df0f6e917f48 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:09.376165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.380286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6267463-7cf4-494c-b725-26293c44ceee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:09.380929 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6267463-7cf4-494c-b725-26293c44ceee service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.381205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6267463-7cf4-494c-b725-26293c44ceee service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.381475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6267463-7cf4-494c-b725-26293c44ceee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.386261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:09.387934 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:09.394345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6267463-7cf4-494c-b725-26293c44ceee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:09.396119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6267463-7cf4-494c-b725-26293c44ceee service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:09.402656 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1268/2533] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:09.407267 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1266/2534] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:10:09.742180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:09.744410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.755291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:09.762194 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbec6b-6a12-4aa5-b02b-31cc2f412d98 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:09.774789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:09.775453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.775771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.776680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] PATH_INFO: `/v3/projects/728ff92e5bef4208bb6e1fc993515e76` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.778103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=728ff92e5bef4208bb6e1fc993515e76)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:09.780318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:09.833967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728ff92e5bef4208bb6e1fc993515e76', 'request_id': 'req-099880bc-4ee1-4f4a-b428-db4c607c13ac'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.845970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728ff92e5bef4208bb6e1fc993515e76', 'request_id': 'req-099880bc-4ee1-4f4a-b428-db4c607c13ac'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.854503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728ff92e5bef4208bb6e1fc993515e76', 'request_id': 'req-099880bc-4ee1-4f4a-b428-db4c607c13ac'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.868205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] The token cache is being invalidate because project 728ff92e5bef4208bb6e1fc993515e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:09.868533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.869270 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-099880bc-4ee1-4f4a-b428-db4c607c13ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:09.871811 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1269/2535] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:09 2022] DELETE /identity/v3/projects/728ff92e5bef4208bb6e1fc993515e76 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:09.876663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.877126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.877245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.897816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:09.899022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.963700 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1267/2536] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:10:09.972609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03109711-426d-4199-9a2f-bef0adf8447d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:09.973131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03109711-426d-4199-9a2f-bef0adf8447d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:09.973131 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03109711-426d-4199-9a2f-bef0adf8447d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:09.995410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:09.995733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-03109711-426d-4199-9a2f-bef0adf8447d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:09.997030 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03109711-426d-4199-9a2f-bef0adf8447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:09.997171 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:10.003551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:10.005183 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1270/2537] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:10.012283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03109711-426d-4199-9a2f-bef0adf8447d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:10.022895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03109711-426d-4199-9a2f-bef0adf8447d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.028520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.029334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.029621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.029893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.030552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:10.030849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.032525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.045895 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1271/2538] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:10:10.050370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:10.061084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f7e44-0eda-4add-834e-e8ab76e8d9ac admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.065849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.066102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86916434-6a81-4a85-a6c9-7e120fb9953f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.066508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.066735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.067011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.067638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.069598 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.069976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.070583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.070864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.071136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.072915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:10.098094 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1272/2539] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:10 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:10:10.102263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:10.141725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.143183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.152938 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1268/2540] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:10:10.173605 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e99c8d-0b37-4649-9dcd-10ab1dfdbea9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.178615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.179230 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.179519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.179793 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.180477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.183267 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.206058 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1273/2541] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:10 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:10:10.210295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:10.221826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf163c1-17bd-425e-9adc-83838cad18b4 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.226595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.227205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.227491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.227765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.228438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.230628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.244407 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1269/2542] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:10.246380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:10.256791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72a3d0-a518-45fd-bf44-41ecec9dfcb4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.261646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f08e1b-79ac-4edb-b9ca-dd4451a6c34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.262258 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f08e1b-79ac-4edb-b9ca-dd4451a6c34d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.262538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f08e1b-79ac-4edb-b9ca-dd4451a6c34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.262813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f08e1b-79ac-4edb-b9ca-dd4451a6c34d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.263462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f08e1b-79ac-4edb-b9ca-dd4451a6c34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.265147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f08e1b-79ac-4edb-b9ca-dd4451a6c34d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.280953 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1274/2543] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:10.282545 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:10.296783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc7789-9cbd-409f-b6c2-535701b8aaa5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.302124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.302803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.303083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.303361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] PATH_INFO: `/v3/projects/21a7731edc664709950a9316fbfb9955/users/6601de6996574ddd877e9dac5697f390/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.331577 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a7731edc664709950a9316fbfb9955, user_id=6601de6996574ddd877e9dac5697f390, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.334348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.355305 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1270/2544] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:10 2022] PUT /identity/v3/projects/21a7731edc664709950a9316fbfb9955/users/6601de6996574ddd877e9dac5697f390/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:10.376105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7a351-6b3d-4374-8f2a-c80363eb2d2b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.376569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7a351-6b3d-4374-8f2a-c80363eb2d2b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.376795 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7a351-6b3d-4374-8f2a-c80363eb2d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.425664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ddd7a351-6b3d-4374-8f2a-c80363eb2d2b None None] MFA Rules not processed for user `6601de6996574ddd877e9dac5697f390`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:10.427402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ddd7a351-6b3d-4374-8f2a-c80363eb2d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:10.450555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:10.533234 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1275/2545] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:10.545753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:10.624945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a93d2-e30f-4952-b0e2-c038f1bab59c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:10.628782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:10.629412 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:10.629693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:10.629964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:10.639220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:10.640681 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:10.656100 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1271/2546] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:14.384064 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7a351-6b3d-4374-8f2a-c80363eb2d2b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:14.397368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7a351-6b3d-4374-8f2a-c80363eb2d2b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:14.401983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249a3fd-6905-4fe4-ad13-c98ce09d8522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:14.402910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a3fd-6905-4fe4-ad13-c98ce09d8522 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:14.403315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a3fd-6905-4fe4-ad13-c98ce09d8522 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:14.403721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a3fd-6905-4fe4-ad13-c98ce09d8522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:14.405853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c249a3fd-6905-4fe4-ad13-c98ce09d8522 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:14.471872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249a3fd-6905-4fe4-ad13-c98ce09d8522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:14.473348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249a3fd-6905-4fe4-ad13-c98ce09d8522 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:14.483106 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1276/2547] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:16.709457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:16.710883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:16.742348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:16.781691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff7fb-cc8b-4d9a-8e44-d93248cc1d79 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:16.787635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:16.788266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:16.788571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:16.788842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] PATH_INFO: `/v3/users/4e339af8d23a4634a2261b6430db3b37` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:16.790090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e339af8d23a4634a2261b6430db3b37)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:16.792075 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:16.842604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e339af8d23a4634a2261b6430db3b37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:16.843277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e339af8d23a4634a2261b6430db3b37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:16.851249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e339af8d23a4634a2261b6430db3b37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:16.863561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e339af8d23a4634a2261b6430db3b37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:16.864278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e339af8d23a4634a2261b6430db3b37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:16.877928 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1272/2548] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:16 2022] DELETE /identity/v3/users/4e339af8d23a4634a2261b6430db3b37 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:16.882668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:16.883181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:16.883278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:16.903945 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:16.905111 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:16.973459 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1277/2549] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:16.984605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:16.986160 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:17.047037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbee88-e90e-4608-87bf-774257d8ef36 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:17.050947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:17.051749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:17.051975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:17.052290 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:17.062440 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:17.063871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:17.074318 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1273/2550] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:17.375936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:17.377757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:17.393787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea3eb5-b40f-4ff0-9906-739a3c4a8270 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:17.403636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:17.404291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:17.404768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:17.404890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] PATH_INFO: `/v3/projects/b9487c7d7abe48fab3402ce97dc5c66f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:17.406136 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9487c7d7abe48fab3402ce97dc5c66f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:17.408341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:17.459516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:17.461680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9487c7d7abe48fab3402ce97dc5c66f', 'request_id': 'req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.469858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9487c7d7abe48fab3402ce97dc5c66f', 'request_id': 'req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.475708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9487c7d7abe48fab3402ce97dc5c66f', 'request_id': 'req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.490226 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] The token cache is being invalidate because project b9487c7d7abe48fab3402ce97dc5c66f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:17.490695 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.491311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.492829 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1278/2551] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:17 2022] DELETE /identity/v3/projects/b9487c7d7abe48fab3402ce97dc5c66f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:17.497569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:17.507192 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d470f7-5322-4d85-b129-866674a78ac4 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:17.576737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:17.593703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:17.594389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:17.594728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:17.595043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] PATH_INFO: `/v3/users/add78d78a2af45e0b47eb43ba02389ac` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:17.596232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add78d78a2af45e0b47eb43ba02389ac)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:17.598330 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:17.655018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add78d78a2af45e0b47eb43ba02389ac'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.655723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add78d78a2af45e0b47eb43ba02389ac'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.664028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add78d78a2af45e0b47eb43ba02389ac'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.678315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add78d78a2af45e0b47eb43ba02389ac'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.678986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add78d78a2af45e0b47eb43ba02389ac'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:17.693715 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1274/2552] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:17 2022] DELETE /identity/v3/users/add78d78a2af45e0b47eb43ba02389ac => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:18.026414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:18.027909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:18.042419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3d5bb-7460-4c17-aec8-5e7c3a6c722a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:18.113278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:18.113926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:18.114211 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:18.114488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] PATH_INFO: `/v3/projects/bfc70072538444f9ad61ae15b984f6f1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:18.115769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc70072538444f9ad61ae15b984f6f1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:18.117988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:18.165928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc70072538444f9ad61ae15b984f6f1', 'request_id': 'req-b737ebef-d294-4c6e-ba74-a072d048b1b8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.172892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc70072538444f9ad61ae15b984f6f1', 'request_id': 'req-b737ebef-d294-4c6e-ba74-a072d048b1b8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.178954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc70072538444f9ad61ae15b984f6f1', 'request_id': 'req-b737ebef-d294-4c6e-ba74-a072d048b1b8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.192253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] The token cache is being invalidate because project bfc70072538444f9ad61ae15b984f6f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:18.192573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.193222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.194726 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1279/2553] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:18 2022] DELETE /identity/v3/projects/bfc70072538444f9ad61ae15b984f6f1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:18.199818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:18.201347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:18.215377 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32859-b88b-4d01-836e-893e8e5c5373 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:18.276686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:18.277370 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:18.277690 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:18.278007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] PATH_INFO: `/v3/users/ad7e019f7a8941ac8b6b3b4a929065c4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:18.279274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7e019f7a8941ac8b6b3b4a929065c4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:18.281447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:18.334718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7e019f7a8941ac8b6b3b4a929065c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.336267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7e019f7a8941ac8b6b3b4a929065c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.343602 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7e019f7a8941ac8b6b3b4a929065c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.354980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:18.358177 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7e019f7a8941ac8b6b3b4a929065c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.359046 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae61aa3a-3ea5-4771-ac81-0aad38c8cf66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7e019f7a8941ac8b6b3b4a929065c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.376787 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1275/2554] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:18 2022] DELETE /identity/v3/users/ad7e019f7a8941ac8b6b3b4a929065c4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:18.671386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:18.673104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:18.690148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ebef-d294-4c6e-ba74-a072d048b1b8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:18.759394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:18.760108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:18.760409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:18.760694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] PATH_INFO: `/v3/projects/c96dc00c9bc146fda6c1b357f45757c0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:18.761881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96dc00c9bc146fda6c1b357f45757c0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:18.763841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:18.772348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:18.819191 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96dc00c9bc146fda6c1b357f45757c0', 'request_id': 'req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.826081 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96dc00c9bc146fda6c1b357f45757c0', 'request_id': 'req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.833050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96dc00c9bc146fda6c1b357f45757c0', 'request_id': 'req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.848461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] The token cache is being invalidate because project c96dc00c9bc146fda6c1b357f45757c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:18.848773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.849485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6471a10e-8d71-4c22-82d1-eb6c53a9c1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:18.851247 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1280/2555] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:18 2022] DELETE /identity/v3/projects/c96dc00c9bc146fda6c1b357f45757c0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:18.858267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3707-2a9a-4943-b667-c4758947967f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:18.858621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3707-2a9a-4943-b667-c4758947967f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:18.858872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3707-2a9a-4943-b667-c4758947967f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:18.879983 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-393c3707-2a9a-4943-b667-c4758947967f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:18.881263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-393c3707-2a9a-4943-b667-c4758947967f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:18.949485 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1276/2556] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:18.958060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896d10a-125c-4e7a-bf1b-c7b1881245fe None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:18.958245 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896d10a-125c-4e7a-bf1b-c7b1881245fe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:18.958614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896d10a-125c-4e7a-bf1b-c7b1881245fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:18.979235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8896d10a-125c-4e7a-bf1b-c7b1881245fe None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:18.980580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8896d10a-125c-4e7a-bf1b-c7b1881245fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:18.989835 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1281/2557] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:18.994357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3707-2a9a-4943-b667-c4758947967f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.004533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3707-2a9a-4943-b667-c4758947967f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.009263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.010414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.010700 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.010977 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.011633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:19.011941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.013638 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.028568 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1277/2558] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:10:19.033149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896d10a-125c-4e7a-bf1b-c7b1881245fe None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.043080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896d10a-125c-4e7a-bf1b-c7b1881245fe None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.048409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.049082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.049368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.062922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.062922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.062922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.084897 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1282/2559] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:19 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:19.090445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.102017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5b10f-a904-4d18-beee-95460bd47229 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.165778 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.166501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.166792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.167080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.167733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.169717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.199012 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1278/2560] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:19 2022] POST /identity/v3/users => generated 301 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:19.200940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.212333 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69ee41-0529-40ea-9435-c09ddd167c36 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.218435 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.219076 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.219353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.219623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.220266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.221915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.235507 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1283/2561] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:19.236665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.249144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50bb6c-2995-42af-aa1c-913b681ab52e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.255387 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f93578-64c3-41fe-8be6-cc2f3c5bd175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.256015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f93578-64c3-41fe-8be6-cc2f3c5bd175 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.256274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f93578-64c3-41fe-8be6-cc2f3c5bd175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.256574 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f93578-64c3-41fe-8be6-cc2f3c5bd175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.257220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f93578-64c3-41fe-8be6-cc2f3c5bd175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.258835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f93578-64c3-41fe-8be6-cc2f3c5bd175 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.277552 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1279/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:19.281853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.295773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c38c-6ae0-4905-ab12-7c87b6876443 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.301504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.302331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.302869 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.303052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] PATH_INFO: `/v3/projects/23d48a7472ab4aeab8eb63e03ecb03ef/users/20fc33e656e641b1a1b3fecbe715721f/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.324587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d48a7472ab4aeab8eb63e03ecb03ef, user_id=20fc33e656e641b1a1b3fecbe715721f, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.328363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.345005 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1284/2563] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:19 2022] PUT /identity/v3/projects/23d48a7472ab4aeab8eb63e03ecb03ef/users/20fc33e656e641b1a1b3fecbe715721f/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:19.362320 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f596a-d64b-46cb-b1c5-09a445411b17 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.362658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f596a-d64b-46cb-b1c5-09a445411b17 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.362879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f596a-d64b-46cb-b1c5-09a445411b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.411843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8e0f596a-d64b-46cb-b1c5-09a445411b17 None None] MFA Rules not processed for user `20fc33e656e641b1a1b3fecbe715721f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:19.414663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e0f596a-d64b-46cb-b1c5-09a445411b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:19.484116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:19.511262 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1280/2564] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:19.515975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.583664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6b728-9728-40d5-9495-72f9e8974263 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.588388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.589012 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.589299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.589578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.590236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.592172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.625175 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1285/2565] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:19 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:19.626442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f596a-d64b-46cb-b1c5-09a445411b17 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.638528 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f596a-d64b-46cb-b1c5-09a445411b17 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.700169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.700812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.701090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.701361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.702031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.704014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.726530 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1281/2566] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:19 2022] POST /identity/v3/users => generated 306 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:19.731095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.742184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf8eea-c597-4542-90d2-4ee3fc10d017 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.747908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.748610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.748896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.749171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.749818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.751519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.762736 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1286/2567] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:19.767235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.777470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cae0ba-8b5f-47cf-81db-8ef003407450 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.782148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.782815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.783077 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.783349 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] PATH_INFO: `/v3/projects/110517209080463ab3cf56a481bafbe0/users/1687e1e938994dd299cee4c85efd98aa/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.799879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110517209080463ab3cf56a481bafbe0, user_id=1687e1e938994dd299cee4c85efd98aa, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.802951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.817442 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1282/2568] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:19 2022] PUT /identity/v3/projects/110517209080463ab3cf56a481bafbe0/users/1687e1e938994dd299cee4c85efd98aa/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:19.821486 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.832004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c72133-b7d9-44a3-8e2d-f63cdc69fb77 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.835811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:19.892633 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8c6fc-0a2e-4c19-aa98-ef60a95bbc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.893228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8c6fc-0a2e-4c19-aa98-ef60a95bbc84 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.893515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8c6fc-0a2e-4c19-aa98-ef60a95bbc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.893783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8c6fc-0a2e-4c19-aa98-ef60a95bbc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.894447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8c6fc-0a2e-4c19-aa98-ef60a95bbc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.896042 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8c6fc-0a2e-4c19-aa98-ef60a95bbc84 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.906693 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1287/2569] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:19.910908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:19.922139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d188efc-6530-4e60-a1ed-095b6d13ed06 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:19.927234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:19.927907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.928185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.928492 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] PATH_INFO: `/v3/projects/110517209080463ab3cf56a481bafbe0/users/1687e1e938994dd299cee4c85efd98aa/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.930372 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=110517209080463ab3cf56a481bafbe0, user_id=1687e1e938994dd299cee4c85efd98aa, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:19.933698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:19.944201 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1283/2570] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:19 2022] PUT /identity/v3/projects/110517209080463ab3cf56a481bafbe0/users/1687e1e938994dd299cee4c85efd98aa/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:19.954821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551f9f1-1f35-4c91-af20-bdd49999c13d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:19.955116 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551f9f1-1f35-4c91-af20-bdd49999c13d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:19.955386 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551f9f1-1f35-4c91-af20-bdd49999c13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:19.984900 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e551f9f1-1f35-4c91-af20-bdd49999c13d None None] MFA Rules not processed for user `1687e1e938994dd299cee4c85efd98aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:19.985529 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e551f9f1-1f35-4c91-af20-bdd49999c13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:20.055917 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1288/2571] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:19 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:20.068499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:20.071008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:20.135631 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8d8b-c26e-4ffb-aebb-086de7a6fa2e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:20.139390 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:20.140007 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:20.140285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:20.140589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:20.150047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:20.152047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:20.161841 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1284/2572] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:20.245589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551f9f1-1f35-4c91-af20-bdd49999c13d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:20.255511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551f9f1-1f35-4c91-af20-bdd49999c13d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:20.259536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5d156-bf45-46d8-8041-8149cee91305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:20.260150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5d156-bf45-46d8-8041-8149cee91305 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:20.260451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5d156-bf45-46d8-8041-8149cee91305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:20.260725 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5d156-bf45-46d8-8041-8149cee91305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:20.329010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5d156-bf45-46d8-8041-8149cee91305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:20.330507 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5d156-bf45-46d8-8041-8149cee91305 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:20.340194 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1289/2573] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:20 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:24.572447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:24.622743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:24.650554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61f272-70c7-4da9-883a-07a5c68cb517 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:24.655437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:24.656062 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:24.656341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:24.656683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] PATH_INFO: `/v3/users/20fc33e656e641b1a1b3fecbe715721f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:24.657868 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fc33e656e641b1a1b3fecbe715721f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:24.659907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:24.710426 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fc33e656e641b1a1b3fecbe715721f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:24.711090 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fc33e656e641b1a1b3fecbe715721f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:24.719450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fc33e656e641b1a1b3fecbe715721f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:24.732998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fc33e656e641b1a1b3fecbe715721f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:24.733720 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fc33e656e641b1a1b3fecbe715721f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:24.747316 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1285/2574] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:24 2022] DELETE /identity/v3/users/20fc33e656e641b1a1b3fecbe715721f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:24.753469 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:24.755124 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:24.755660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:24.902168 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:24.903299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:24.924843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:24.967856 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1290/2575] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:24.978642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:24.980205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:25.042632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd032f67-c135-403a-aa48-f3644d2ecbf5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:25.046169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:25.046778 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:25.047047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:25.047320 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:25.057414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:25.058846 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:25.068996 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1286/2576] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:25.374595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:25.376138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:25.392529 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608a02c-bc56-45ac-9a8a-14067701d201 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:25.398280 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:25.398926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:25.399242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:25.399478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] PATH_INFO: `/v3/projects/23d48a7472ab4aeab8eb63e03ecb03ef` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:25.400811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d48a7472ab4aeab8eb63e03ecb03ef)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:25.402691 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:25.458873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d48a7472ab4aeab8eb63e03ecb03ef', 'request_id': 'req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.465994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d48a7472ab4aeab8eb63e03ecb03ef', 'request_id': 'req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.472028 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d48a7472ab4aeab8eb63e03ecb03ef', 'request_id': 'req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.484954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] The token cache is being invalidate because project 23d48a7472ab4aeab8eb63e03ecb03ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:25.485242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.485929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.487450 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1291/2577] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:25 2022] DELETE /identity/v3/projects/23d48a7472ab4aeab8eb63e03ecb03ef => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:25.491762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:25.502600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e177c4-09bd-4316-87d1-a9d7ee61ea52 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:25.563968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:25.564656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:25.564895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:25.565166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] PATH_INFO: `/v3/users/1687e1e938994dd299cee4c85efd98aa` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:25.566287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1687e1e938994dd299cee4c85efd98aa)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:25.568225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:25.569880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:25.625494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1687e1e938994dd299cee4c85efd98aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.626772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1687e1e938994dd299cee4c85efd98aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.635453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1687e1e938994dd299cee4c85efd98aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.647991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1687e1e938994dd299cee4c85efd98aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.648774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41ee7ee8-0ccb-447d-99b2-85d14002d3a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1687e1e938994dd299cee4c85efd98aa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:25.663713 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1287/2578] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:25 2022] DELETE /identity/v3/users/1687e1e938994dd299cee4c85efd98aa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:25.963670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:25.965259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:25.978628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068f7c7-bfbe-4844-9018-e6d537c0b4b2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.046563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.047236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.047558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.047827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] PATH_INFO: `/v3/projects/110517209080463ab3cf56a481bafbe0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.048992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=110517209080463ab3cf56a481bafbe0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.050896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.104249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110517209080463ab3cf56a481bafbe0', 'request_id': 'req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:26.107409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:26.114451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110517209080463ab3cf56a481bafbe0', 'request_id': 'req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:26.121407 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110517209080463ab3cf56a481bafbe0', 'request_id': 'req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:26.135177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] The token cache is being invalidate because project 110517209080463ab3cf56a481bafbe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:26.135472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:26.136102 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7b096e5a-ca4f-45b6-845f-54ee9a2e8fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:26.137618 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1292/2579] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:25 2022] DELETE /identity/v3/projects/110517209080463ab3cf56a481bafbe0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:26.144826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3d386-7e7b-4305-82a6-725dcd530134 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.145110 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3d386-7e7b-4305-82a6-725dcd530134 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.145383 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3d386-7e7b-4305-82a6-725dcd530134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.169705 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d2f3d386-7e7b-4305-82a6-725dcd530134 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:26.171018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d2f3d386-7e7b-4305-82a6-725dcd530134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:26.230969 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1288/2580] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:26.240217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f242-1937-41a0-8dd3-a4a7a26ba959 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.240544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f242-1937-41a0-8dd3-a4a7a26ba959 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.240845 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f242-1937-41a0-8dd3-a4a7a26ba959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.260341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0080f242-1937-41a0-8dd3-a4a7a26ba959 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:26.261604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0080f242-1937-41a0-8dd3-a4a7a26ba959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:26.269529 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1293/2581] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:10:26.274063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3d386-7e7b-4305-82a6-725dcd530134 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.284879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3d386-7e7b-4305-82a6-725dcd530134 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.289718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.290315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.290684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.291505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.292148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:26.292474 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.294049 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.306047 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1289/2582] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:10:26.314056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f242-1937-41a0-8dd3-a4a7a26ba959 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.323447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f242-1937-41a0-8dd3-a4a7a26ba959 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.328252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.328903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.329227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.329525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.330155 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.332319 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.361858 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1294/2583] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:26 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:26.366341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.376403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237cac4-37b5-47bd-9914-87652a67d8de admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.434509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.435109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.435385 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.435654 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.436279 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.438203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.450310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:26.467742 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1290/2584] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:26 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:26.469282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.480728 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0d69e-5977-487d-a49a-84c16f6eccd6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.496027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.496715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.496998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.497271 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.497925 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.499591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.510878 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1295/2585] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:26.514864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.525827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d73332-9aca-4a5a-ad8e-fa1e139bfd3a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.530637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac270219-0d1f-453e-9b58-a2aeccd61bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.531235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac270219-0d1f-453e-9b58-a2aeccd61bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.531517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac270219-0d1f-453e-9b58-a2aeccd61bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.531811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac270219-0d1f-453e-9b58-a2aeccd61bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.532467 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac270219-0d1f-453e-9b58-a2aeccd61bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.534064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac270219-0d1f-453e-9b58-a2aeccd61bd0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.545998 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1291/2586] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:26.550907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.561852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d548-4315-4be3-bf43-f624d5685d48 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.567351 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.567801 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.568149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.568490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] PATH_INFO: `/v3/projects/90c360feead942318a90716408a25f33/users/6819d80640e848f19251795685afae0f/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.589020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c360feead942318a90716408a25f33, user_id=6819d80640e848f19251795685afae0f, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.593457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.609281 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1296/2587] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:26 2022] PUT /identity/v3/projects/90c360feead942318a90716408a25f33/users/6819d80640e848f19251795685afae0f/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:26.620671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8534d-34c5-4970-8e19-314f28ca673b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.620955 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8534d-34c5-4970-8e19-314f28ca673b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.621221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8534d-34c5-4970-8e19-314f28ca673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.650362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dbf8534d-34c5-4970-8e19-314f28ca673b None None] MFA Rules not processed for user `6819d80640e848f19251795685afae0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:26.651587 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dbf8534d-34c5-4970-8e19-314f28ca673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:26.719002 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1292/2588] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:26.724138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.788960 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c523b-28a5-4843-adb9-6f455d0c547f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.794887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.795526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.795796 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.796068 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.796731 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.799209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.829661 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1297/2589] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:26 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:26.833783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8534d-34c5-4970-8e19-314f28ca673b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.846669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8534d-34c5-4970-8e19-314f28ca673b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.908606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.909240 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.909518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.909790 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.910441 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.912677 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.939433 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1293/2590] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:26 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:26.944429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.956294 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b6615-5f48-4b29-89c4-e719ba5b4aa3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:26.963509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:26.964217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:26.964806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:26.964806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:26.966866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:26.968854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:26.980682 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1298/2591] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:26.985004 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:26.995260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eab86-b861-4a95-be80-a3ae8f5e2ade admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:27.000101 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:27.000800 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:27.001084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:27.001355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] PATH_INFO: `/v3/projects/64175d1115f249b382f70e8284652e96/users/e7a0f7b2ea854fa490b1b6cc4c3928f4/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:27.019094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64175d1115f249b382f70e8284652e96, user_id=e7a0f7b2ea854fa490b1b6cc4c3928f4, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:27.022261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:27.037671 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1294/2592] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:26 2022] PUT /identity/v3/projects/64175d1115f249b382f70e8284652e96/users/e7a0f7b2ea854fa490b1b6cc4c3928f4/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:27.042791 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:27.053307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc90e6-293b-4d01-8553-43d4c47a0d1d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:27.102878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:27.116077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213116c-24fb-4906-ac10-0102520ced41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:27.116727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213116c-24fb-4906-ac10-0102520ced41 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:27.117013 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213116c-24fb-4906-ac10-0102520ced41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:27.117287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213116c-24fb-4906-ac10-0102520ced41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:27.117953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213116c-24fb-4906-ac10-0102520ced41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:27.119594 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213116c-24fb-4906-ac10-0102520ced41 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:27.130318 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1299/2593] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:27.134582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:27.145352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32033a6c-8b00-48b7-b833-60970b9485b5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:27.150897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:27.151655 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:27.151942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:27.152224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] PATH_INFO: `/v3/projects/64175d1115f249b382f70e8284652e96/users/e7a0f7b2ea854fa490b1b6cc4c3928f4/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:27.154495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64175d1115f249b382f70e8284652e96, user_id=e7a0f7b2ea854fa490b1b6cc4c3928f4, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:27.157256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:27.168885 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1295/2594] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:27 2022] PUT /identity/v3/projects/64175d1115f249b382f70e8284652e96/users/e7a0f7b2ea854fa490b1b6cc4c3928f4/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:27.179619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4925a3-2048-40e5-97e5-27b602388a54 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:27.179880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4925a3-2048-40e5-97e5-27b602388a54 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:27.180159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4925a3-2048-40e5-97e5-27b602388a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:27.209135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8f4925a3-2048-40e5-97e5-27b602388a54 None None] MFA Rules not processed for user `e7a0f7b2ea854fa490b1b6cc4c3928f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:27.210355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8f4925a3-2048-40e5-97e5-27b602388a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:27.286466 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1300/2595] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:27.297761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:27.299470 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:27.332733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:27.364757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ea5ac-e80f-461b-80e8-04838c43a2e0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:27.368434 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028e5ac-967e-4a42-aedb-b3a0795c02e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:27.369041 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028e5ac-967e-4a42-aedb-b3a0795c02e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:27.369322 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028e5ac-967e-4a42-aedb-b3a0795c02e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:27.369585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028e5ac-967e-4a42-aedb-b3a0795c02e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:27.378984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028e5ac-967e-4a42-aedb-b3a0795c02e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:27.380396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028e5ac-967e-4a42-aedb-b3a0795c02e2 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:27.391435 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1296/2596] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:28.007325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4925a3-2048-40e5-97e5-27b602388a54 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:28.073200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4925a3-2048-40e5-97e5-27b602388a54 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:28.078517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:28.079430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:28.079713 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:28.080309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] PATH_INFO: `/v3/users/6819d80640e848f19251795685afae0f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:28.081490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6819d80640e848f19251795685afae0f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:28.083766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:28.141300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6819d80640e848f19251795685afae0f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.148992 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6819d80640e848f19251795685afae0f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.162655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6819d80640e848f19251795685afae0f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.163376 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6819d80640e848f19251795685afae0f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.164058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6819d80640e848f19251795685afae0f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.166587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:28.178605 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1301/2597] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:28 2022] DELETE /identity/v3/users/6819d80640e848f19251795685afae0f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:28.183777 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:28.184502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:28.184502 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:28.204329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:28.205591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:28.266748 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1297/2598] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:28.276661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:28.278143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:28.346244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237c45f-c724-4ef2-b575-fd58df4b0572 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:28.349794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:28.350403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:28.350694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:28.350966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:28.361289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:28.362744 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:28.373099 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1302/2599] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:28.681098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:28.682575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:28.695391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068e7b-f5b0-4a46-a3b0-d992a4dff800 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:28.702837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:28.703617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:28.703948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:28.704293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] PATH_INFO: `/v3/projects/90c360feead942318a90716408a25f33` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:28.705625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c360feead942318a90716408a25f33)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:28.707692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:28.765921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c360feead942318a90716408a25f33', 'request_id': 'req-9faba670-be36-484c-bb9b-8051738ac1b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.774896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c360feead942318a90716408a25f33', 'request_id': 'req-9faba670-be36-484c-bb9b-8051738ac1b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.785184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c360feead942318a90716408a25f33', 'request_id': 'req-9faba670-be36-484c-bb9b-8051738ac1b7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.797498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] The token cache is being invalidate because project 90c360feead942318a90716408a25f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:28.797780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.798430 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.799888 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1298/2600] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:28 2022] DELETE /identity/v3/projects/90c360feead942318a90716408a25f33 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:28.803893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:28.814367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810922e4-045a-41f7-8593-6e71b730d9fb service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:28.877557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:28.878073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:28.878363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:28.878632 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] PATH_INFO: `/v3/users/e7a0f7b2ea854fa490b1b6cc4c3928f4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:28.879799 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a0f7b2ea854fa490b1b6cc4c3928f4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:28.881746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:28.930291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a0f7b2ea854fa490b1b6cc4c3928f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.938024 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a0f7b2ea854fa490b1b6cc4c3928f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.951309 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a0f7b2ea854fa490b1b6cc4c3928f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.953250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a0f7b2ea854fa490b1b6cc4c3928f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.953250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df0b743a-1284-4d9c-9698-529d342e2749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a0f7b2ea854fa490b1b6cc4c3928f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:28.965001 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1303/2601] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:28 2022] DELETE /identity/v3/users/e7a0f7b2ea854fa490b1b6cc4c3928f4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:29.251586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:29.253420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:29.270984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba670-be36-484c-bb9b-8051738ac1b7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:29.332551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:29.344774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:29.345427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.345856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.346347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] PATH_INFO: `/v3/projects/64175d1115f249b382f70e8284652e96` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.347663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64175d1115f249b382f70e8284652e96)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:29.349611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:29.404949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64175d1115f249b382f70e8284652e96', 'request_id': 'req-de4d09ad-3a86-48a9-be17-15a4ed6260b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:29.414299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64175d1115f249b382f70e8284652e96', 'request_id': 'req-de4d09ad-3a86-48a9-be17-15a4ed6260b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:29.420951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64175d1115f249b382f70e8284652e96', 'request_id': 'req-de4d09ad-3a86-48a9-be17-15a4ed6260b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:29.432883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] The token cache is being invalidate because project 64175d1115f249b382f70e8284652e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:29.433169 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:29.433824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-de4d09ad-3a86-48a9-be17-15a4ed6260b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:29.435330 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1299/2602] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:29 2022] DELETE /identity/v3/projects/64175d1115f249b382f70e8284652e96 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:29.440283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c95acd-8fc0-4422-ad3e-d464d7376ceb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.440879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c95acd-8fc0-4422-ad3e-d464d7376ceb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.440879 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c95acd-8fc0-4422-ad3e-d464d7376ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.461366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c4c95acd-8fc0-4422-ad3e-d464d7376ceb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:29.463148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c4c95acd-8fc0-4422-ad3e-d464d7376ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:29.471566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:29.528632 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1304/2603] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:29.594035 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4ec4c-5b5a-4e28-9bca-f23fa664a208 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.594328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4ec4c-5b5a-4e28-9bca-f23fa664a208 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.594605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4ec4c-5b5a-4e28-9bca-f23fa664a208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.614597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-35b4ec4c-5b5a-4e28-9bca-f23fa664a208 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:29.615817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-35b4ec4c-5b5a-4e28-9bca-f23fa664a208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:29.623901 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1300/2604] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:29.628282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c95acd-8fc0-4422-ad3e-d464d7376ceb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:29.639619 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c95acd-8fc0-4422-ad3e-d464d7376ceb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:29.644614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:29.645239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.645526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.645803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.646459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:29.646766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:29.648401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:29.661434 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1305/2605] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:10:29.667081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4ec4c-5b5a-4e28-9bca-f23fa664a208 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:29.679435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4ec4c-5b5a-4e28-9bca-f23fa664a208 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:29.684481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:29.685163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.685446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.685710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.686338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:29.688471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:29.717327 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1301/2606] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:29 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:29.721775 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:29.732818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d326f-d9e7-43b7-9281-cfafa1a76b10 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:29.794142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:29.794778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.795047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.795324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.795973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:29.797955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:29.820763 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1306/2607] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:29 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:29.825960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:29.836590 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db813b-3fb1-421f-afbe-09e960696193 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:29.841882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:29.842512 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.842797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.843088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.843729 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:29.845395 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:29.856088 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1302/2608] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:29.860936 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:29.871576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3a500-f460-45b3-98a3-07df928604f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:29.876781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bdfc6-1497-495b-9f34-bda8c0d32a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:29.877391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bdfc6-1497-495b-9f34-bda8c0d32a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.877664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bdfc6-1497-495b-9f34-bda8c0d32a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.877930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bdfc6-1497-495b-9f34-bda8c0d32a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.878570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bdfc6-1497-495b-9f34-bda8c0d32a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:29.880167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bdfc6-1497-495b-9f34-bda8c0d32a12 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:29.891755 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1307/2609] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:29.896255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:29.906618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ef5f0-ef62-4f90-91e7-d5f4972a10e7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:29.911315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:29.912184 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.912483 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.912757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] PATH_INFO: `/v3/projects/b1eb7cc474724d59be53e5ce749c346d/users/b2334ab9b86249d1ae1854081aa59107/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.927618 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1eb7cc474724d59be53e5ce749c346d, user_id=b2334ab9b86249d1ae1854081aa59107, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:29.930295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:29.945123 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1303/2610] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:29 2022] PUT /identity/v3/projects/b1eb7cc474724d59be53e5ce749c346d/users/b2334ab9b86249d1ae1854081aa59107/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:29.955750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a040d-cfa6-4b08-a7a4-d85ba1171871 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:29.956054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a040d-cfa6-4b08-a7a4-d85ba1171871 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:29.956338 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a040d-cfa6-4b08-a7a4-d85ba1171871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:29.987405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-560a040d-cfa6-4b08-a7a4-d85ba1171871 None None] MFA Rules not processed for user `b2334ab9b86249d1ae1854081aa59107`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:29.988642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-560a040d-cfa6-4b08-a7a4-d85ba1171871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:30.054925 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1308/2611] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:30.066724 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:30.068924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:30.127731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c956cf9-e10d-4706-8239-6d7e24d1fd86 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:30.131602 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f267743-f1e5-4468-b889-c87e7134dc2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:30.132225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267743-f1e5-4468-b889-c87e7134dc2a service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:30.132546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267743-f1e5-4468-b889-c87e7134dc2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:30.132816 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f267743-f1e5-4468-b889-c87e7134dc2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:30.142521 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f267743-f1e5-4468-b889-c87e7134dc2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:30.143951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f267743-f1e5-4468-b889-c87e7134dc2a service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:30.152819 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1304/2612] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:30.173956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a040d-cfa6-4b08-a7a4-d85ba1171871 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:30.185075 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:30.240920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a040d-cfa6-4b08-a7a4-d85ba1171871 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:30.246243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:30.246904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:30.247186 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:30.247462 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] PATH_INFO: `/v3/users/b2334ab9b86249d1ae1854081aa59107` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:30.248631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2334ab9b86249d1ae1854081aa59107)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:30.250559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:30.300139 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2334ab9b86249d1ae1854081aa59107'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.309571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2334ab9b86249d1ae1854081aa59107'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.323999 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2334ab9b86249d1ae1854081aa59107'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.324802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2334ab9b86249d1ae1854081aa59107'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.325452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2334ab9b86249d1ae1854081aa59107'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.339693 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1309/2613] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:30 2022] DELETE /identity/v3/users/b2334ab9b86249d1ae1854081aa59107 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:30.344225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:30.345150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:30.345150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:30.364513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:30.365732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:30.383197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:30.430255 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1305/2614] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:30.441069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:30.442571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:30.505645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038154c-f1be-4779-aa7d-7bc8cd885d7c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:30.509474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae992164-228a-425d-b4c5-fc8e08787948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:30.510128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae992164-228a-425d-b4c5-fc8e08787948 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:30.510445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae992164-228a-425d-b4c5-fc8e08787948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:30.510724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae992164-228a-425d-b4c5-fc8e08787948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:30.521154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae992164-228a-425d-b4c5-fc8e08787948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:30.522621 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae992164-228a-425d-b4c5-fc8e08787948 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:30.532493 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1310/2615] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:30.841787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:30.843498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:30.859934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88706f1a-02b0-44e8-8362-6562c4e67294 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:30.864918 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:30.865651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:30.865939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:30.866215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] PATH_INFO: `/v3/projects/b1eb7cc474724d59be53e5ce749c346d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:30.867421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1eb7cc474724d59be53e5ce749c346d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:30.869516 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:30.917914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1eb7cc474724d59be53e5ce749c346d', 'request_id': 'req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.926572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1eb7cc474724d59be53e5ce749c346d', 'request_id': 'req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.935273 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1eb7cc474724d59be53e5ce749c346d', 'request_id': 'req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.946611 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] The token cache is being invalidate because project b1eb7cc474724d59be53e5ce749c346d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:30.946917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.947580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a49d6adb-8bbe-4514-ac1a-757284ce1bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:30.949097 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1306/2616] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:30 2022] DELETE /identity/v3/projects/b1eb7cc474724d59be53e5ce749c346d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:30.954252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d9fd-38f8-4bc5-b28d-748cbcef5050 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:30.954541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d9fd-38f8-4bc5-b28d-748cbcef5050 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:30.954818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d9fd-38f8-4bc5-b28d-748cbcef5050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:30.975353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-62f7d9fd-38f8-4bc5-b28d-748cbcef5050 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:30.976860 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62f7d9fd-38f8-4bc5-b28d-748cbcef5050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:31.006094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:31.044061 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1311/2617] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:31.051759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1179-8b68-4337-8f13-b84db50353b9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.052043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1179-8b68-4337-8f13-b84db50353b9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.052318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1179-8b68-4337-8f13-b84db50353b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.073142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c4eb1179-8b68-4337-8f13-b84db50353b9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:31.074394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c4eb1179-8b68-4337-8f13-b84db50353b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:31.082335 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1307/2618] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:31.086819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d9fd-38f8-4bc5-b28d-748cbcef5050 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:31.096952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d9fd-38f8-4bc5-b28d-748cbcef5050 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:31.101880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:31.102623 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.103520 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.103908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.104928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:31.105354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:31.107388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:31.120250 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1312/2619] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:10:31.124922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1179-8b68-4337-8f13-b84db50353b9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:31.134484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1179-8b68-4337-8f13-b84db50353b9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:31.139588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:31.140211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.140534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.140812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.141468 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:31.143365 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:31.173852 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1308/2620] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:31 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:31.177901 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:31.189470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760255e-3e09-41ca-8460-fe8ced16b3d2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:31.257149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:31.257769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.258041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.258310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.258950 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:31.260966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:31.284350 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1313/2621] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:31 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:10:31.288285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:31.299670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04cce1-4574-43d8-8bc1-e8d604dfcb31 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:31.304422 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:31.305050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.305327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.305599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.306889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:31.308533 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:31.321723 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1309/2622] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:31.325814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:31.336754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded2c4c-0422-4808-9e75-bf8728ae8a93 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:31.342595 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd009f-9832-4d60-8821-d0c5b85ce94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:31.343264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd009f-9832-4d60-8821-d0c5b85ce94e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.343542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd009f-9832-4d60-8821-d0c5b85ce94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.343816 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd009f-9832-4d60-8821-d0c5b85ce94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.344497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd009f-9832-4d60-8821-d0c5b85ce94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:31.346106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd009f-9832-4d60-8821-d0c5b85ce94e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:31.358370 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1314/2623] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:31.363448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:31.375159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed42e7-9e6d-4875-aa51-ceb6b147a3f5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:31.380220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:31.380945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.381223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.381507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] PATH_INFO: `/v3/projects/f0619fcd95094b57a7657d6d33744a5d/users/e10fce587d244b29913f4a75e8f71e72/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.399460 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0619fcd95094b57a7657d6d33744a5d, user_id=e10fce587d244b29913f4a75e8f71e72, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:31.402645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:31.418278 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1310/2624] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:31 2022] PUT /identity/v3/projects/f0619fcd95094b57a7657d6d33744a5d/users/e10fce587d244b29913f4a75e8f71e72/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:31.429830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cde6e-703a-4c3a-b4fb-ec0f5effe188 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.430118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cde6e-703a-4c3a-b4fb-ec0f5effe188 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.430394 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cde6e-703a-4c3a-b4fb-ec0f5effe188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.459939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-390cde6e-703a-4c3a-b4fb-ec0f5effe188 None None] MFA Rules not processed for user `e10fce587d244b29913f4a75e8f71e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:31.461132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-390cde6e-703a-4c3a-b4fb-ec0f5effe188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:31.531717 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1315/2625] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:10:31.544287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:31.546266 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:31.595547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:31.613125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4057277-8148-4e6e-8b77-1a55e765d43c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:31.617905 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa55ff-4b34-41e4-bc99-fb5ecbe0c0ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:31.618910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa55ff-4b34-41e4-bc99-fb5ecbe0c0ba service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:31.619209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa55ff-4b34-41e4-bc99-fb5ecbe0c0ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:31.619479 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa55ff-4b34-41e4-bc99-fb5ecbe0c0ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:31.628793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa55ff-4b34-41e4-bc99-fb5ecbe0c0ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:31.630319 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa55ff-4b34-41e4-bc99-fb5ecbe0c0ba service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:31.640993 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1311/2626] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:34.743297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cde6e-703a-4c3a-b4fb-ec0f5effe188 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:34.773989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:34.808893 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cde6e-703a-4c3a-b4fb-ec0f5effe188 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:34.813322 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:34.813954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:34.814238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:34.814517 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] PATH_INFO: `/v3/users/e10fce587d244b29913f4a75e8f71e72` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:34.815646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10fce587d244b29913f4a75e8f71e72)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:34.817609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:34.866883 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10fce587d244b29913f4a75e8f71e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:34.874617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10fce587d244b29913f4a75e8f71e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:34.887148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10fce587d244b29913f4a75e8f71e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:34.887833 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10fce587d244b29913f4a75e8f71e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:34.888458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10fce587d244b29913f4a75e8f71e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:34.901026 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1316/2627] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:34 2022] DELETE /identity/v3/users/e10fce587d244b29913f4a75e8f71e72 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:34.905579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:34.906059 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:34.906205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:34.925622 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:34.926871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:34.988015 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1312/2628] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:34.997814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:34.999305 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:35.059032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596667cd-d5f1-4159-8604-9d79876423f2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.071332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b9658-cd28-4c1f-99fa-e57dc5462829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.071965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9658-cd28-4c1f-99fa-e57dc5462829 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.072233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9658-cd28-4c1f-99fa-e57dc5462829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.072534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9658-cd28-4c1f-99fa-e57dc5462829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.088067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b9658-cd28-4c1f-99fa-e57dc5462829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.089551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b9658-cd28-4c1f-99fa-e57dc5462829 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.101035 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1317/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:35.385748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:35.387484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:35.401401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10db2c-87b7-4745-8b80-0a937473ce1c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.406744 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.407403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.407736 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.408011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] PATH_INFO: `/v3/projects/f0619fcd95094b57a7657d6d33744a5d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.409225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0619fcd95094b57a7657d6d33744a5d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.411991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.462459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0619fcd95094b57a7657d6d33744a5d', 'request_id': 'req-df91fd10-14a6-4f39-81cf-eca2066f1c82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:35.471519 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0619fcd95094b57a7657d6d33744a5d', 'request_id': 'req-df91fd10-14a6-4f39-81cf-eca2066f1c82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:35.477473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0619fcd95094b57a7657d6d33744a5d', 'request_id': 'req-df91fd10-14a6-4f39-81cf-eca2066f1c82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:35.489283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] The token cache is being invalidate because project f0619fcd95094b57a7657d6d33744a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:35.489580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:35.490222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df91fd10-14a6-4f39-81cf-eca2066f1c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:35.491650 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1313/2630] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:35 2022] DELETE /identity/v3/projects/f0619fcd95094b57a7657d6d33744a5d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:35.497433 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96b7ea-77b1-4aad-8732-6adb6d2c0a85 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.497782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96b7ea-77b1-4aad-8732-6adb6d2c0a85 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.498126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96b7ea-77b1-4aad-8732-6adb6d2c0a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.518844 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1a96b7ea-77b1-4aad-8732-6adb6d2c0a85 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:35.520072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1a96b7ea-77b1-4aad-8732-6adb6d2c0a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:35.566298 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:35.583381 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1318/2631] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:35.591493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b020-2e6b-4f78-b4b1-7d94650bac6d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.591776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b020-2e6b-4f78-b4b1-7d94650bac6d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.592060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b020-2e6b-4f78-b4b1-7d94650bac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.612297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ce38b020-2e6b-4f78-b4b1-7d94650bac6d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:35.613549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ce38b020-2e6b-4f78-b4b1-7d94650bac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:35.622180 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1314/2632] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:35.626527 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96b7ea-77b1-4aad-8732-6adb6d2c0a85 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:35.636406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96b7ea-77b1-4aad-8732-6adb6d2c0a85 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.640990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.641585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.641868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.642146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.642818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:10:35.643146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.644766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.656604 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1319/2633] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:10:35.660872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b020-2e6b-4f78-b4b1-7d94650bac6d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:35.670826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b020-2e6b-4f78-b4b1-7d94650bac6d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.675551 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.676161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.676481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.676756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.677389 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.679291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.707422 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1315/2634] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:10:35 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:35.711873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:35.722011 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7cdc2-3bbb-4122-9f26-6263011c9799 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.779680 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.780441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.780550 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.780821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.781438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.783363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.805576 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1320/2635] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:10:35 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:10:35.810342 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:35.820621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ceebf2-ae59-4f82-a24d-d93812a97672 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.825117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.825710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.825986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.826256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.826893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.828501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.838901 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1316/2636] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:35.842963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:35.854113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19418f-e38c-4a67-957c-9bdd2c29b027 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.859219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e59f-40e5-43f2-9f47-9a6bb656a91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.859822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e59f-40e5-43f2-9f47-9a6bb656a91f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.860107 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e59f-40e5-43f2-9f47-9a6bb656a91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.860404 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e59f-40e5-43f2-9f47-9a6bb656a91f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.861056 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e59f-40e5-43f2-9f47-9a6bb656a91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.862698 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e59f-40e5-43f2-9f47-9a6bb656a91f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.873914 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1321/2637] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:10:35.878136 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:35.888312 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d9fba-25d3-4519-9e09-ac6b0e811fd3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:35.892957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:35.893584 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.893866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.894134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] PATH_INFO: `/v3/projects/18a552c30f85444b88e70aad53c40058/users/828083ee56c440d5a3e659021d2104b0/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.909142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a552c30f85444b88e70aad53c40058, user_id=828083ee56c440d5a3e659021d2104b0, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:35.912068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:35.928616 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1317/2638] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:10:35 2022] PUT /identity/v3/projects/18a552c30f85444b88e70aad53c40058/users/828083ee56c440d5a3e659021d2104b0/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:35.934263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80420a69-e784-4144-90f3-be8223ede825 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:35.934547 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80420a69-e784-4144-90f3-be8223ede825 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:35.934823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80420a69-e784-4144-90f3-be8223ede825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:35.955776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-80420a69-e784-4144-90f3-be8223ede825 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:35.957046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80420a69-e784-4144-90f3-be8223ede825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:36.023854 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1322/2639] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:36.034580 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:36.036366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:36.045333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:36.104408 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898005a-a486-4429-8e1e-5c70b2ce925e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:36.108043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:36.108673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:36.108962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:36.109239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:36.118326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:36.119756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:36.129309 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1318/2640] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:36.803454 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8de3-d852-43e1-85be-54e20e04e21e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:36.804043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8de3-d852-43e1-85be-54e20e04e21e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:36.804300 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8de3-d852-43e1-85be-54e20e04e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:36.833876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-748d8de3-d852-43e1-85be-54e20e04e21e None None] MFA Rules not processed for user `828083ee56c440d5a3e659021d2104b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:36.835095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-748d8de3-d852-43e1-85be-54e20e04e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:36.883774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:36.909773 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1323/2641] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:36.920784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:36.922227 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:36.996911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3209968-cade-43c7-8c9f-f1053231369b service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:37.001104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:37.001799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:37.002112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:37.002416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:37.012337 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:37.013867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:37.022981 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1319/2642] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:38.392761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09929fd7-4030-4491-9764-03301797727c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:38.393372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09929fd7-4030-4491-9764-03301797727c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:38.393372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09929fd7-4030-4491-9764-03301797727c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:38.416159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-09929fd7-4030-4491-9764-03301797727c None None] MFA Rules not processed for user `828083ee56c440d5a3e659021d2104b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:38.417556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-09929fd7-4030-4491-9764-03301797727c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:38.424615 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1324/2643] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:38.470676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:38.472089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:38.531553 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee1fd4-f656-439d-8e6f-f6e685e53d62 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:38.535188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:38.535793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:38.536068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:38.536341 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:38.538363 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:38.596613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:38.611586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:38.612986 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:38.624309 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1320/2644] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:39.150339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09929fd7-4030-4491-9764-03301797727c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:39.151890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-09929fd7-4030-4491-9764-03301797727c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:39.164269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09929fd7-4030-4491-9764-03301797727c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:39.170609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202e3c7-7ead-4f0e-9a8b-5829b9cb19c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:39.171225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202e3c7-7ead-4f0e-9a8b-5829b9cb19c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:39.172126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202e3c7-7ead-4f0e-9a8b-5829b9cb19c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:39.172441 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202e3c7-7ead-4f0e-9a8b-5829b9cb19c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:39.174287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6202e3c7-7ead-4f0e-9a8b-5829b9cb19c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:39.195980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202e3c7-7ead-4f0e-9a8b-5829b9cb19c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:39.198096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202e3c7-7ead-4f0e-9a8b-5829b9cb19c6 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:39.212430 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1325/2645] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:47.462862 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:47.464427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:47.478111 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433119f-6eeb-47ad-8a56-84bf9acbe850 service glance] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:47.483559 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:47.484197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:47.484508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:47.484786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] PATH_INFO: `/v3/users/6601de6996574ddd877e9dac5697f390` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:47.485939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6601de6996574ddd877e9dac5697f390)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:47.487887 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:47.542297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6601de6996574ddd877e9dac5697f390'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:47.543120 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6601de6996574ddd877e9dac5697f390'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:47.551756 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6601de6996574ddd877e9dac5697f390'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:47.568750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6601de6996574ddd877e9dac5697f390'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:47.569570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6601de6996574ddd877e9dac5697f390'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:47.585697 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1321/2646] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:10:47 2022] DELETE /identity/v3/users/6601de6996574ddd877e9dac5697f390 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:10:47.590622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:47.591207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:47.591402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:47.621002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:47.622272 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:47.687259 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1326/2647] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:10:47.698268 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:47.699874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:47.757892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd874b-011a-4691-88f3-99a3618ec972 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:47.762037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0ef3c-c464-4872-b77f-24bb658d618d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:47.762636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0ef3c-c464-4872-b77f-24bb658d618d service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:47.762915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0ef3c-c464-4872-b77f-24bb658d618d service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:47.763209 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0ef3c-c464-4872-b77f-24bb658d618d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:47.772314 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0ef3c-c464-4872-b77f-24bb658d618d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:47.773766 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0ef3c-c464-4872-b77f-24bb658d618d service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:47.783811 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1322/2648] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:10:48.053782 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:10:48.056031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:48.068975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879be7d-a0d1-401a-aad9-ad763b3644b7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:10:48.074806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:10:48.075681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:48.075973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:48.076243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] PATH_INFO: `/v3/projects/21a7731edc664709950a9316fbfb9955` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:48.079785 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a7731edc664709950a9316fbfb9955)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:10:48.082048 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:10:48.132083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a7731edc664709950a9316fbfb9955', 'request_id': 'req-04485b16-ce24-4d4d-86ff-2ca73ff77924'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:48.138732 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a7731edc664709950a9316fbfb9955', 'request_id': 'req-04485b16-ce24-4d4d-86ff-2ca73ff77924'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:48.144697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a7731edc664709950a9316fbfb9955', 'request_id': 'req-04485b16-ce24-4d4d-86ff-2ca73ff77924'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:48.157555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] The token cache is being invalidate because project 21a7731edc664709950a9316fbfb9955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:10:48.157875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:48.158601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:10:48.160199 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1327/2649] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:10:48 2022] DELETE /identity/v3/projects/21a7731edc664709950a9316fbfb9955 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:10:56.539523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:10:56.540038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:10:56.540038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:10:56.559964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] MFA Rules not processed for user `828083ee56c440d5a3e659021d2104b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:10:56.561210 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:10:56.621859 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1323/2650] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:10:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:10:58.154274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:06.614605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:08.971110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:08.972722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:09.043274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485b16-ce24-4d4d-86ff-2ca73ff77924 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:09.048584 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:09.049474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:09.049884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:09.050291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:09.052800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:09.127926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:09.129593 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:09.140890 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1328/2651] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:13.021976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:13.024312 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:13.041636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9af5f-2db3-43ef-8b28-761ac535569f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:13.046461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:13.047251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.047529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.047804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] PATH_INFO: `/v3/users/c468aa6d1248478286fe0a8792411a91` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.064619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c468aa6d1248478286fe0a8792411a91)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:13.066612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:13.128529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c468aa6d1248478286fe0a8792411a91'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.129228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c468aa6d1248478286fe0a8792411a91'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.137920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c468aa6d1248478286fe0a8792411a91'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.152626 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c468aa6d1248478286fe0a8792411a91'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.153449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d82ea0e6-e1e0-4c22-aa50-774c638e0acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c468aa6d1248478286fe0a8792411a91'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.169108 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1324/2652] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:11:13 2022] DELETE /identity/v3/users/c468aa6d1248478286fe0a8792411a91 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:13.366099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:13.377511 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d6353-ebb4-4458-bd54-8f003906c8c5 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:13.473683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:13.474264 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.474553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.474823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] PATH_INFO: `/v3/projects/cc51ee9f630a4bc89f1fccbd7ff20c51` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.488420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc51ee9f630a4bc89f1fccbd7ff20c51)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:13.490434 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:13.542187 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:13.547617 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc51ee9f630a4bc89f1fccbd7ff20c51', 'request_id': 'req-717fdac7-f76c-450c-aed8-938092e44046'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.555474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc51ee9f630a4bc89f1fccbd7ff20c51', 'request_id': 'req-717fdac7-f76c-450c-aed8-938092e44046'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.562062 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc51ee9f630a4bc89f1fccbd7ff20c51', 'request_id': 'req-717fdac7-f76c-450c-aed8-938092e44046'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.575719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] The token cache is being invalidate because project cc51ee9f630a4bc89f1fccbd7ff20c51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:11:13.576094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.576787 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-717fdac7-f76c-450c-aed8-938092e44046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:13.578378 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1329/2653] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:11:13 2022] DELETE /identity/v3/projects/cc51ee9f630a4bc89f1fccbd7ff20c51 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:13.584127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9432409-d457-4da7-bc8f-745ad7ac22c2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.584717 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9432409-d457-4da7-bc8f-745ad7ac22c2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.584939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9432409-d457-4da7-bc8f-745ad7ac22c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.607547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c9432409-d457-4da7-bc8f-745ad7ac22c2 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:13.608920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c9432409-d457-4da7-bc8f-745ad7ac22c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:13.671632 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1325/2654] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:13.680363 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a0429-b649-4c1b-ae7f-f4c7bc6836ef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.680689 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a0429-b649-4c1b-ae7f-f4c7bc6836ef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.680965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a0429-b649-4c1b-ae7f-f4c7bc6836ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.703954 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4d6a0429-b649-4c1b-ae7f-f4c7bc6836ef None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:13.705296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4d6a0429-b649-4c1b-ae7f-f4c7bc6836ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:13.714266 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1330/2655] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:13.719113 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9432409-d457-4da7-bc8f-745ad7ac22c2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:13.729442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9432409-d457-4da7-bc8f-745ad7ac22c2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:13.734280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:13.734914 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.735202 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.735473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.736142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:11:13.736625 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:13.738306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:13.756234 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1326/2656] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:11:13.762189 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a0429-b649-4c1b-ae7f-f4c7bc6836ef None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:13.773515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a0429-b649-4c1b-ae7f-f4c7bc6836ef None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:13.778838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:13.779604 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.779901 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.780180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.780866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:13.782876 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:13.812955 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1331/2657] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:11:13 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:13.817373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:13.828579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe183f-112c-4911-8f9c-b0f26bf3964e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:13.875219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:13.890291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:13.891019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.891197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.891471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.892121 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:13.894123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:13.918960 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1327/2658] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:11:13 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:13.923345 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:13.939284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bfc64-e8e1-46d8-a6ea-16d206f2295e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:13.944095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:13.945239 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.945516 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.945788 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.946431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:13.948041 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:13.959345 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1332/2659] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:13.964144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:13.974409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983c80d-776e-4d5c-85ac-74152af665dd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:13.981052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20693cbe-84f4-4025-98d8-d4c6d0add435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:13.981798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20693cbe-84f4-4025-98d8-d4c6d0add435 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:13.982128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20693cbe-84f4-4025-98d8-d4c6d0add435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:13.982560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20693cbe-84f4-4025-98d8-d4c6d0add435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:13.983510 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20693cbe-84f4-4025-98d8-d4c6d0add435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:13.985797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20693cbe-84f4-4025-98d8-d4c6d0add435 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:13.999565 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1328/2660] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:11:14.006561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:14.018373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3792999-c166-4f8c-92f4-220a26bcff0c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:14.023822 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:14.024568 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:14.024872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:14.025145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] PATH_INFO: `/v3/projects/d0628ae5b7904b21958d59e93e659a17/users/8622855b67c045409adc0dfa52ce06b5/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:14.043764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0628ae5b7904b21958d59e93e659a17, user_id=8622855b67c045409adc0dfa52ce06b5, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:14.046528 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:14.062009 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1333/2661] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:14 2022] PUT /identity/v3/projects/d0628ae5b7904b21958d59e93e659a17/users/8622855b67c045409adc0dfa52ce06b5/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:14.074315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a694-7ab5-4470-b8ba-9025dced6cda None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:14.074917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a694-7ab5-4470-b8ba-9025dced6cda None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:14.074917 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a694-7ab5-4470-b8ba-9025dced6cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:14.109414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d128a694-7ab5-4470-b8ba-9025dced6cda None None] MFA Rules not processed for user `8622855b67c045409adc0dfa52ce06b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:14.110672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d128a694-7ab5-4470-b8ba-9025dced6cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:14.182479 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1329/2662] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:11:14.196018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:14.198113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:14.287762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf274-1110-477a-ac62-085cd1ddee99 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:14.296165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca046d-424e-49d2-9d69-4a4244ca5f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:14.296839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca046d-424e-49d2-9d69-4a4244ca5f8f service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:14.297128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca046d-424e-49d2-9d69-4a4244ca5f8f service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:14.297405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca046d-424e-49d2-9d69-4a4244ca5f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:14.307939 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca046d-424e-49d2-9d69-4a4244ca5f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:14.311225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca046d-424e-49d2-9d69-4a4244ca5f8f service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:14.325193 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1334/2663] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:11:17.115135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a694-7ab5-4470-b8ba-9025dced6cda None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:17.184562 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a694-7ab5-4470-b8ba-9025dced6cda None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:17.190902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:17.191663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:17.191979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:17.192258 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] PATH_INFO: `/v3/users/8622855b67c045409adc0dfa52ce06b5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:17.193592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8622855b67c045409adc0dfa52ce06b5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:17.195601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:17.256356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8622855b67c045409adc0dfa52ce06b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:17.257658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8622855b67c045409adc0dfa52ce06b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:17.265275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8622855b67c045409adc0dfa52ce06b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:17.267663 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:17.281837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8622855b67c045409adc0dfa52ce06b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:17.282784 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8622855b67c045409adc0dfa52ce06b5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:17.303166 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1330/2664] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:11:17 2022] DELETE /identity/v3/users/8622855b67c045409adc0dfa52ce06b5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:17.311292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:17.311940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:17.311940 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:17.342577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:17.344145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:17.405364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:17.423659 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1335/2665] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:17.434888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:17.437473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:17.535086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3f47b-f338-43a8-99fc-cda8d19da6c0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:17.540664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48778d0-e436-4094-9c32-ec20e79e3200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:17.541321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48778d0-e436-4094-9c32-ec20e79e3200 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:17.541601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48778d0-e436-4094-9c32-ec20e79e3200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:17.541957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48778d0-e436-4094-9c32-ec20e79e3200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:17.555278 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48778d0-e436-4094-9c32-ec20e79e3200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:17.556988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48778d0-e436-4094-9c32-ec20e79e3200 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:17.567633 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1331/2666] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:17.891910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:17.893408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:17.907665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6b83a-c179-4751-96b5-1e7e69081517 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:17.914509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:17.915427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:17.915756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:17.916078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] PATH_INFO: `/v3/projects/d0628ae5b7904b21958d59e93e659a17` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:17.917805 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0628ae5b7904b21958d59e93e659a17)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:17.920636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:17.978518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0628ae5b7904b21958d59e93e659a17', 'request_id': 'req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:17.985919 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0628ae5b7904b21958d59e93e659a17', 'request_id': 'req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:17.994659 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0628ae5b7904b21958d59e93e659a17', 'request_id': 'req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:18.012746 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] The token cache is being invalidate because project d0628ae5b7904b21958d59e93e659a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:11:18.013069 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:18.013839 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9578e905-5e9c-4a81-ab2f-5d0d3e5f24db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:18.018713 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1336/2667] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:11:17 2022] DELETE /identity/v3/projects/d0628ae5b7904b21958d59e93e659a17 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:18.024326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b971a-4078-4c56-af88-3e57e9970af9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.024761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b971a-4078-4c56-af88-3e57e9970af9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.025143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b971a-4078-4c56-af88-3e57e9970af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.046392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0f5b971a-4078-4c56-af88-3e57e9970af9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:18.047660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f5b971a-4078-4c56-af88-3e57e9970af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:18.123839 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1332/2668] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:18.133168 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75bfce-8907-48c3-a789-a501fa04e09e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.133168 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75bfce-8907-48c3-a789-a501fa04e09e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.133168 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75bfce-8907-48c3-a789-a501fa04e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.155397 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ee75bfce-8907-48c3-a789-a501fa04e09e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:18.156716 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ee75bfce-8907-48c3-a789-a501fa04e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:18.167146 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1337/2669] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:11:18.171807 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b971a-4078-4c56-af88-3e57e9970af9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.181798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b971a-4078-4c56-af88-3e57e9970af9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.186400 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.187008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.187283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.187552 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.188197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:11:18.188523 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.190112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.202912 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1333/2670] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:11:18.207566 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75bfce-8907-48c3-a789-a501fa04e09e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.218033 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75bfce-8907-48c3-a789-a501fa04e09e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.225461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.226078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.226387 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.226657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.227327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.229296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.258709 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1338/2671] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:18.263005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.273604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe6d4-6a2e-49af-9db0-7204a591c4ac admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.333500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.334123 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.334405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.334794 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.335332 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.337450 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.350237 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:18.366077 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1334/2672] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:11:18.371559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.385032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54379eeb-c6ff-4189-85dd-44b882abd709 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.391694 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.393203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.393916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.394706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.395870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.397762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.418187 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1339/2673] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:18.420836 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.432560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f0674-2cdc-4f3f-bea9-bc38748f4558 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.438646 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba1c9d-2b55-4945-9c6b-d04623c763c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.439267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba1c9d-2b55-4945-9c6b-d04623c763c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.439567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba1c9d-2b55-4945-9c6b-d04623c763c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.439842 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba1c9d-2b55-4945-9c6b-d04623c763c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.440644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba1c9d-2b55-4945-9c6b-d04623c763c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.442423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba1c9d-2b55-4945-9c6b-d04623c763c3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.456097 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1335/2674] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:18.460519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.464566 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954aa1-1812-455f-ab4a-0af8aab283e4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.465084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954aa1-1812-455f-ab4a-0af8aab283e4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.465461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4954aa1-1812-455f-ab4a-0af8aab283e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.472296 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900cc2-8ed0-411e-be19-c15864b780a3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.478419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037167e5-cf43-491b-a171-e8f8c65484f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.479195 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037167e5-cf43-491b-a171-e8f8c65484f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.479479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037167e5-cf43-491b-a171-e8f8c65484f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.479756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037167e5-cf43-491b-a171-e8f8c65484f5 admin admin] PATH_INFO: `/v3/projects/312fe7bfc7b3458e8673890853b437f8/users/86ead2c31610436db5282f841a5fcde3/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.497370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037167e5-cf43-491b-a171-e8f8c65484f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312fe7bfc7b3458e8673890853b437f8, user_id=86ead2c31610436db5282f841a5fcde3, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.500329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037167e5-cf43-491b-a171-e8f8c65484f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.516675 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1340/2675] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:18 2022] PUT /identity/v3/projects/312fe7bfc7b3458e8673890853b437f8/users/86ead2c31610436db5282f841a5fcde3/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:18.528480 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c4954aa1-1812-455f-ab4a-0af8aab283e4 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:18.529768 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c4954aa1-1812-455f-ab4a-0af8aab283e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:18.529917 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.530147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.530427 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.561941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] MFA Rules not processed for user `86ead2c31610436db5282f841a5fcde3`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:18.563250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:18.610251 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1336/2676] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:11:18.649738 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1341/2677] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:11:18.654749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4954aa1-1812-455f-ab4a-0af8aab283e4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.665292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.667047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:18.670667 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:18.755115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4954aa1-1812-455f-ab4a-0af8aab283e4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.761627 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.762257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.762539 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.762815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.764302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.765636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.804561 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1337/2678] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:18.807256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.823556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f0c37e-9c09-48b1-8dea-fc0d3f225331 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.862707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:18.884843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eb731-f23f-4df9-ae6e-f5d1c5049a75 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.891403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.892712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.893000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.893325 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.893975 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.895919 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.900778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.901432 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.901803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.902110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.919414 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1338/2679] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:11:18 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:18.923243 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.934323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe40c6-da49-4215-8529-4a5511131160 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.940916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.940916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.940916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.940916 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.943063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.943300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.954522 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1339/2680] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:18.958948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:18.968916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:18.969118 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499092c-0571-43b6-9e3e-bdd94e6eb95e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:18.970539 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:18.973990 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:18.974662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:18.974940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:18.975233 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] PATH_INFO: `/v3/projects/9a3db69aa5ef49cc9322b3360180a0d8/users/0841379c5aa747d591180c4e69d2e3e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:18.980704 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1342/2681] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:11:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 25 01:11:18.996890 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3db69aa5ef49cc9322b3360180a0d8, user_id=0841379c5aa747d591180c4e69d2e3e8, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:19.001031 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:19.026797 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1340/2682] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:18 2022] PUT /identity/v3/projects/9a3db69aa5ef49cc9322b3360180a0d8/users/0841379c5aa747d591180c4e69d2e3e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:19.031101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:19.048581 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebaf17-bbf8-414a-8206-ec4b0145d368 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:19.077491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:19.149827 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ff31a-6ba1-458d-a9bc-9fbe04a7203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:19.150912 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ff31a-6ba1-458d-a9bc-9fbe04a7203f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:19.151419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ff31a-6ba1-458d-a9bc-9fbe04a7203f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:19.151915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ff31a-6ba1-458d-a9bc-9fbe04a7203f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:19.153180 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ff31a-6ba1-458d-a9bc-9fbe04a7203f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:19.156344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ff31a-6ba1-458d-a9bc-9fbe04a7203f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:19.175136 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1343/2683] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:19.180745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:19.193364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dd6a2-05c7-484f-a961-f6987eb34942 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:19.200641 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:19.201339 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:19.201621 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:19.201899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] PATH_INFO: `/v3/projects/9a3db69aa5ef49cc9322b3360180a0d8/users/0841379c5aa747d591180c4e69d2e3e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:19.204472 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3db69aa5ef49cc9322b3360180a0d8, user_id=0841379c5aa747d591180c4e69d2e3e8, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:19.207102 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:19.224606 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1341/2684] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:19 2022] PUT /identity/v3/projects/9a3db69aa5ef49cc9322b3360180a0d8/users/0841379c5aa747d591180c4e69d2e3e8/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:19.238891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2110ec-b95a-409c-b200-d7914dd8a5a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:19.239213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2110ec-b95a-409c-b200-d7914dd8a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:19.239436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2110ec-b95a-409c-b200-d7914dd8a5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:19.277715 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ba2110ec-b95a-409c-b200-d7914dd8a5a4 None None] MFA Rules not processed for user `0841379c5aa747d591180c4e69d2e3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:19.279053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba2110ec-b95a-409c-b200-d7914dd8a5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:19.374072 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1344/2685] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:11:19.386526 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:19.389949 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:19.465188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74021bfb-178c-46d2-97ea-3d2ad4c9afb9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:19.471380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:19.472050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:19.472329 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:19.472636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:19.483364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:19.484852 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:19.503067 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1342/2686] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:19.619577 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2110ec-b95a-409c-b200-d7914dd8a5a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:19.635843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2110ec-b95a-409c-b200-d7914dd8a5a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:19.639948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06902a7c-f19e-44a0-95ea-c1c3271be4bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:19.640612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06902a7c-f19e-44a0-95ea-c1c3271be4bf service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:19.640894 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06902a7c-f19e-44a0-95ea-c1c3271be4bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:19.641165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06902a7c-f19e-44a0-95ea-c1c3271be4bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:19.737963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06902a7c-f19e-44a0-95ea-c1c3271be4bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:19.739671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06902a7c-f19e-44a0-95ea-c1c3271be4bf service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:19.754105 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1345/2687] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:22.594486 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:22.596256 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:22.679967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708b22b-0be9-434a-9b10-82b708dd4314 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:22.685709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:22.686547 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:22.686955 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:22.687453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:22.693158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:22.721581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:22.723880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:22.749145 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1343/2688] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:29.746161 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=79 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:32.744564 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:36.273032 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:36.273551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:36.273655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:36.294652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:36.295896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:36.358879 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1346/2689] 10.222.0.26 () {58 vars in 967 bytes} [Sun Sep 25 01:11:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:11:36.371900 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:36.443361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b436a-8666-444a-81ac-fbc99fd0c83c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:36.448248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:36.448892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:36.449173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:36.449446 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:36.458628 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:36.460292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:36.469291 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1344/2690] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:11:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:11:38.482498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:38.483846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:38.557103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20875eda-d3ba-4f40-816a-4651dc5367d8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:38.561709 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:38.562334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:38.562615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:38.562895 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] PATH_INFO: `/v3/users/828083ee56c440d5a3e659021d2104b0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:38.564083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828083ee56c440d5a3e659021d2104b0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:38.566061 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:38.617288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828083ee56c440d5a3e659021d2104b0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:38.628128 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828083ee56c440d5a3e659021d2104b0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:38.644608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828083ee56c440d5a3e659021d2104b0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:38.645613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828083ee56c440d5a3e659021d2104b0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:38.647213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828083ee56c440d5a3e659021d2104b0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:38.661348 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1347/2691] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:11:38 2022] DELETE /identity/v3/users/828083ee56c440d5a3e659021d2104b0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:38.849130 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:38.873739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4ca69-ea75-4010-90ed-2c4d4986a355 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:38.940984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:38.941872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:38.942157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:38.946295 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] PATH_INFO: `/v3/projects/18a552c30f85444b88e70aad53c40058` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:38.946805 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a552c30f85444b88e70aad53c40058)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:38.950274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:39.000589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a552c30f85444b88e70aad53c40058', 'request_id': 'req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:39.009058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a552c30f85444b88e70aad53c40058', 'request_id': 'req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:39.015219 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a552c30f85444b88e70aad53c40058', 'request_id': 'req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:39.027415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] The token cache is being invalidate because project 18a552c30f85444b88e70aad53c40058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:11:39.027725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:39.028491 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dfd2dd21-07f6-4a6c-8f06-217bc8ca7ddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:39.030191 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1345/2692] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:11:38 2022] DELETE /identity/v3/projects/18a552c30f85444b88e70aad53c40058 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:48.657676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:49.024082 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:52.680439 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:52.681929 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:52.696970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090ec5b-905b-4142-8cc9-33c5d0334019 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:52.758225 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:52.758926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:52.759207 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:52.759483 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] PATH_INFO: `/v3/users/86ead2c31610436db5282f841a5fcde3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:52.760742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ead2c31610436db5282f841a5fcde3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:52.763077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:52.831701 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ead2c31610436db5282f841a5fcde3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:52.842887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ead2c31610436db5282f841a5fcde3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:52.863071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ead2c31610436db5282f841a5fcde3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:52.863990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ead2c31610436db5282f841a5fcde3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:52.864838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ead2c31610436db5282f841a5fcde3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:52.887771 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1348/2693] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:11:52 2022] DELETE /identity/v3/users/86ead2c31610436db5282f841a5fcde3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:52.891224 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:52.892284 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:52.892586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:52.917912 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:52.919715 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:52.997020 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1346/2694] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:53.007302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:53.008784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:53.081222 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6aa74-5adc-408a-a220-b1c6d15cd833 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:53.086599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:53.087254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:53.087537 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:53.087810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:53.099324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:53.100847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:53.112883 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1349/2695] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:53.437054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:53.438751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:53.453672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924b35d-42c4-4ae0-b6a1-229b39798f20 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:53.458830 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:53.459445 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:53.459727 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:53.460002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] PATH_INFO: `/v3/projects/312fe7bfc7b3458e8673890853b437f8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:53.461165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=312fe7bfc7b3458e8673890853b437f8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:53.463071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:53.513168 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312fe7bfc7b3458e8673890853b437f8', 'request_id': 'req-6c6ae754-362f-46c7-b5c9-949e5734ec5c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.522116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312fe7bfc7b3458e8673890853b437f8', 'request_id': 'req-6c6ae754-362f-46c7-b5c9-949e5734ec5c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.528659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312fe7bfc7b3458e8673890853b437f8', 'request_id': 'req-6c6ae754-362f-46c7-b5c9-949e5734ec5c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.541083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] The token cache is being invalidate because project 312fe7bfc7b3458e8673890853b437f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:11:53.541366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.542036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.543499 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1347/2696] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:11:53 2022] DELETE /identity/v3/projects/312fe7bfc7b3458e8673890853b437f8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:53.547522 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:53.558434 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9b78-7bb6-4c12-a8f6-63ea442abd87 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:53.576564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:53.618858 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:53.619586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:53.619872 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:53.620148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] PATH_INFO: `/v3/users/0841379c5aa747d591180c4e69d2e3e8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:53.621356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0841379c5aa747d591180c4e69d2e3e8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:53.623318 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:53.672031 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0841379c5aa747d591180c4e69d2e3e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.680085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0841379c5aa747d591180c4e69d2e3e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.693118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0841379c5aa747d591180c4e69d2e3e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.693859 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0841379c5aa747d591180c4e69d2e3e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.694508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e8446bcd-6d8c-4f83-83fc-0ce5cf64ecb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0841379c5aa747d591180c4e69d2e3e8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:53.707899 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1350/2697] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:11:53 2022] DELETE /identity/v3/users/0841379c5aa747d591180c4e69d2e3e8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:54.001698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.003229 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:54.017693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ae754-362f-46c7-b5c9-949e5734ec5c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.084704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.085352 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.085632 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.085907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] PATH_INFO: `/v3/projects/9a3db69aa5ef49cc9322b3360180a0d8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.087058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3db69aa5ef49cc9322b3360180a0d8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.089030 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.123028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:54.142396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3db69aa5ef49cc9322b3360180a0d8', 'request_id': 'req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:54.151966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3db69aa5ef49cc9322b3360180a0d8', 'request_id': 'req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:54.158191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3db69aa5ef49cc9322b3360180a0d8', 'request_id': 'req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:54.171262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] The token cache is being invalidate because project 9a3db69aa5ef49cc9322b3360180a0d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:11:54.171505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:54.172154 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20ccfea8-e77f-41a5-8b6e-9aa7b494857e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:54.173662 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1348/2698] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:11:54 2022] DELETE /identity/v3/projects/9a3db69aa5ef49cc9322b3360180a0d8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:54.179281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd154c7-d086-4e9c-8051-448ec41a92d7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.179615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd154c7-d086-4e9c-8051-448ec41a92d7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.179892 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd154c7-d086-4e9c-8051-448ec41a92d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.203078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9dd154c7-d086-4e9c-8051-448ec41a92d7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:54.204536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9dd154c7-d086-4e9c-8051-448ec41a92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:54.287639 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1351/2699] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:54.294371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863363-9b47-4cf3-96c6-a6271af1ff3d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.294683 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863363-9b47-4cf3-96c6-a6271af1ff3d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.294959 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93863363-9b47-4cf3-96c6-a6271af1ff3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.318163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-93863363-9b47-4cf3-96c6-a6271af1ff3d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:54.319449 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-93863363-9b47-4cf3-96c6-a6271af1ff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:54.328611 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1349/2700] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:54.334080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd154c7-d086-4e9c-8051-448ec41a92d7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.349047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd154c7-d086-4e9c-8051-448ec41a92d7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.355249 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.356029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.356165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.356434 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.357100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:11:54.357402 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.359027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.372677 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1352/2701] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 25 01:11:54.377674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93863363-9b47-4cf3-96c6-a6271af1ff3d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.389020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93863363-9b47-4cf3-96c6-a6271af1ff3d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.393984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.394605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.394906 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.395170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.396515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.399188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.431501 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1350/2702] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:11:54 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:54.435278 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.447077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeff6c9-a00e-42a9-be6a-7ff631dfd911 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.478101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:54.517880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.517880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.517880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.517880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.518575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.520634 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.543815 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1353/2703] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:11:54 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:54.548354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.564877 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfbb37-6489-43f8-9f13-95b87d74f193 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.570860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.571225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.571507 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.571780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.572471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.574099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.595685 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1351/2704] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:54.600256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.613763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc96591-02db-4d50-b969-772a277c8bd3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.620092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cea38-d889-4428-be5f-ff5293b7377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.620747 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cea38-d889-4428-be5f-ff5293b7377f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.621060 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cea38-d889-4428-be5f-ff5293b7377f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.621323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cea38-d889-4428-be5f-ff5293b7377f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.621975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cea38-d889-4428-be5f-ff5293b7377f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.623616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cea38-d889-4428-be5f-ff5293b7377f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.644435 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1354/2705] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:54.646115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.656498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20fdb4-2998-4c7a-9ffd-4cc210c3cc65 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.661513 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.662176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.662457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.662733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] PATH_INFO: `/v3/projects/ffc72637d719428f9f243766d9901224/users/ea264005274f42eaa75ba89a84d1a0d6/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.678981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc72637d719428f9f243766d9901224, user_id=ea264005274f42eaa75ba89a84d1a0d6, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.681695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.696269 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1352/2706] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:54 2022] PUT /identity/v3/projects/ffc72637d719428f9f243766d9901224/users/ea264005274f42eaa75ba89a84d1a0d6/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:54.708451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19085f9-e4db-4f45-a675-c835379bc906 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.709182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19085f9-e4db-4f45-a675-c835379bc906 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.709182 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19085f9-e4db-4f45-a675-c835379bc906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.740726 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a19085f9-e4db-4f45-a675-c835379bc906 None None] MFA Rules not processed for user `ea264005274f42eaa75ba89a84d1a0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:54.741949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a19085f9-e4db-4f45-a675-c835379bc906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:54.838298 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1355/2707] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:54.843203 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.911354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613bfee-1340-4c37-8ff6-0dfe01e38eab admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:54.916092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:54.916726 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:54.917011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:54.917286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:54.917915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:54.919835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:54.949038 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1353/2708] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:11:54 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:54.953417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19085f9-e4db-4f45-a675-c835379bc906 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:54.965485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19085f9-e4db-4f45-a675-c835379bc906 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:55.032027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:55.032761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.033120 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.033525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.034159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:55.037364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:55.061223 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1356/2709] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:11:54 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:55.065404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:55.076510 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f3d5-4b99-42aa-ae84-c6d02c682c81 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:55.081707 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:55.082331 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.082671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.082953 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.083629 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:55.085261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:55.096976 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1354/2710] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:55.102436 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:55.113655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9dbfd-977b-4895-a604-327fdc073a0e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:55.119022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:55.119686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.119962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.120233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] PATH_INFO: `/v3/projects/ab50e96b6b874a2fa22cbfb8a8150064/users/47505ac801d042f6b5852be04a06ee88/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.136544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab50e96b6b874a2fa22cbfb8a8150064, user_id=47505ac801d042f6b5852be04a06ee88, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:55.139184 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:55.152932 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1357/2711] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:55 2022] PUT /identity/v3/projects/ab50e96b6b874a2fa22cbfb8a8150064/users/47505ac801d042f6b5852be04a06ee88/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:55.156970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:55.170810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4374f-13fe-4f6d-9079-b42644ef45ac admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:55.193459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:55.235835 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6df532-a6de-4834-9f52-9e5cce8726ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:55.236481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6df532-a6de-4834-9f52-9e5cce8726ee admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.236761 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6df532-a6de-4834-9f52-9e5cce8726ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.237032 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6df532-a6de-4834-9f52-9e5cce8726ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.237673 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6df532-a6de-4834-9f52-9e5cce8726ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:55.239333 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6df532-a6de-4834-9f52-9e5cce8726ee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:55.250627 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1355/2712] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:55.254834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:55.261167 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:55.267603 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33806c-8a4a-480c-819a-300ef5c93789 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:55.273849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:55.274440 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.274719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.274995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] PATH_INFO: `/v3/projects/ab50e96b6b874a2fa22cbfb8a8150064/users/47505ac801d042f6b5852be04a06ee88/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.277307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab50e96b6b874a2fa22cbfb8a8150064, user_id=47505ac801d042f6b5852be04a06ee88, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:55.280026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:55.291232 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1358/2713] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:55 2022] PUT /identity/v3/projects/ab50e96b6b874a2fa22cbfb8a8150064/users/47505ac801d042f6b5852be04a06ee88/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:55.301501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.302358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.302630 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.336753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] MFA Rules not processed for user `47505ac801d042f6b5852be04a06ee88`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:55.338458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:55.414290 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1356/2714] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:55.426636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:55.429244 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:55.500129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51dd33c-7142-4d4a-b318-5704652f9754 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:55.504231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:55.504921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.505735 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.506009 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.517385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:55.520415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:55.531038 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1359/2715] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:55.628005 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:55.629911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:55.646134 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315d5bd-66c9-43cf-b60d-3e06cd0838be None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:55.650871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35635062-af51-47a5-ae11-123f0c37e0c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:55.651718 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635062-af51-47a5-ae11-123f0c37e0c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:55.652190 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635062-af51-47a5-ae11-123f0c37e0c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:55.652713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635062-af51-47a5-ae11-123f0c37e0c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:55.726467 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35635062-af51-47a5-ae11-123f0c37e0c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:55.727926 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35635062-af51-47a5-ae11-123f0c37e0c2 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:55.736955 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1357/2716] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:56.048597 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:56.118368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e36a24-4318-4655-b847-a54db9ae2058 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:56.124373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:56.125400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:56.125679 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:56.125953 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] PATH_INFO: `/v3/users/ea264005274f42eaa75ba89a84d1a0d6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:56.127482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea264005274f42eaa75ba89a84d1a0d6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:56.129531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:56.187234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea264005274f42eaa75ba89a84d1a0d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.195388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea264005274f42eaa75ba89a84d1a0d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.210846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea264005274f42eaa75ba89a84d1a0d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.211576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea264005274f42eaa75ba89a84d1a0d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.212290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea264005274f42eaa75ba89a84d1a0d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.229447 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1360/2717] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:11:56 2022] DELETE /identity/v3/users/ea264005274f42eaa75ba89a84d1a0d6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:56.234388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:56.234943 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:56.235087 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:56.255144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:56.256563 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:56.301930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:56.327372 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1358/2718] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:56.339188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:56.340861 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:56.357456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:56.425914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d98132-3788-401b-be00-efa106c739f0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:56.429956 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:56.430586 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:56.430878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:56.431152 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:56.441381 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:56.443311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:56.454031 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1361/2719] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:11:56.838188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:56.840135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:56.863053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b0a99-83dd-4cdb-977a-e83001edc25e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:56.869844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:56.870098 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:56.870404 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:56.872262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] PATH_INFO: `/v3/projects/ffc72637d719428f9f243766d9901224` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:56.874245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc72637d719428f9f243766d9901224)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:56.875773 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:56.879745 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:56.882631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:56.899106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646bf2-2f5c-4b4b-a1d9-8ab95658ff0a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:56.905662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:56.906677 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:56.907169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:56.907564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:56.909505 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:56.925163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:56.928084 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:56.945071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc72637d719428f9f243766d9901224', 'request_id': 'req-15365fc9-95db-4de1-8928-69aae0501e7e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.946947 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1362/2720] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:11:56.955808 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc72637d719428f9f243766d9901224', 'request_id': 'req-15365fc9-95db-4de1-8928-69aae0501e7e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.965645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc72637d719428f9f243766d9901224', 'request_id': 'req-15365fc9-95db-4de1-8928-69aae0501e7e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.983964 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] The token cache is being invalidate because project ffc72637d719428f9f243766d9901224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:11:56.984301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.985000 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:56.987024 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1359/2721] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:11:56 2022] DELETE /identity/v3/projects/ffc72637d719428f9f243766d9901224 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:56.991065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:57.001941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360412e8-f716-498c-a99c-474567f92110 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:57.097281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:57.098054 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:57.098336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:57.098620 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] PATH_INFO: `/v3/users/47505ac801d042f6b5852be04a06ee88` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:57.099915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=47505ac801d042f6b5852be04a06ee88)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:57.101891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:57.182639 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47505ac801d042f6b5852be04a06ee88'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.196951 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47505ac801d042f6b5852be04a06ee88'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.215323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47505ac801d042f6b5852be04a06ee88'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.216237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47505ac801d042f6b5852be04a06ee88'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.217012 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a528388b-1fc4-4b60-ab41-b5acb49e5d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47505ac801d042f6b5852be04a06ee88'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.230380 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1363/2722] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:11:56 2022] DELETE /identity/v3/users/47505ac801d042f6b5852be04a06ee88 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:57.572216 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:57.574880 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:57.591850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15365fc9-95db-4de1-8928-69aae0501e7e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:57.669585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:57.669996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:57.670418 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:57.670871 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] PATH_INFO: `/v3/projects/ab50e96b6b874a2fa22cbfb8a8150064` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:57.672841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab50e96b6b874a2fa22cbfb8a8150064)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:57.675454 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:57.742072 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab50e96b6b874a2fa22cbfb8a8150064', 'request_id': 'req-5669ac69-333e-4455-9ddc-fd0a966bd8c0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.755226 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab50e96b6b874a2fa22cbfb8a8150064', 'request_id': 'req-5669ac69-333e-4455-9ddc-fd0a966bd8c0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.762589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab50e96b6b874a2fa22cbfb8a8150064', 'request_id': 'req-5669ac69-333e-4455-9ddc-fd0a966bd8c0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.777152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:57.787544 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] The token cache is being invalidate because project ab50e96b6b874a2fa22cbfb8a8150064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:11:57.788131 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.789850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5669ac69-333e-4455-9ddc-fd0a966bd8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:11:57.795086 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1360/2723] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:11:57 2022] DELETE /identity/v3/projects/ab50e96b6b874a2fa22cbfb8a8150064 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:11:57.799339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470cdba-313c-4bbb-b5b2-d49794d61efc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:57.799741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470cdba-313c-4bbb-b5b2-d49794d61efc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:57.800073 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470cdba-313c-4bbb-b5b2-d49794d61efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:57.822674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7470cdba-313c-4bbb-b5b2-d49794d61efc None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:57.823509 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7470cdba-313c-4bbb-b5b2-d49794d61efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:57.844233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:57.933651 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1364/2724] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:11:57.944217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfad71-238c-4ad0-893e-2f262f25357d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:57.944686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfad71-238c-4ad0-893e-2f262f25357d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:57.945089 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfad71-238c-4ad0-893e-2f262f25357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:57.970283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-83cfad71-238c-4ad0-893e-2f262f25357d None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:57.971828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-83cfad71-238c-4ad0-893e-2f262f25357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:57.981235 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1361/2725] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:57.987453 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470cdba-313c-4bbb-b5b2-d49794d61efc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:58.005821 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470cdba-313c-4bbb-b5b2-d49794d61efc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:58.011428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:58.012301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.012983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.012983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.013773 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:11:58.014080 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:58.015774 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:58.034440 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1365/2726] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:11:58.040186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfad71-238c-4ad0-893e-2f262f25357d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:58.051116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfad71-238c-4ad0-893e-2f262f25357d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:58.057876 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:58.058506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.058782 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.059084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.059695 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:58.061843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:58.093892 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1362/2727] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:11:58 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:58.097972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:58.110470 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0e044-7867-4982-9a5a-2c8a7b86611f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:58.186247 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:58.187196 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.187389 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.187660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.188477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:58.190823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:58.220633 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1366/2728] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:11:58 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:11:58.222186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:58.234112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b40bd-4363-4f1a-987d-ac8eaef88488 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:58.241421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:58.241969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.242251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.242529 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.243277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:58.245132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:58.265627 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1363/2729] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:58.267179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:58.280916 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356f0f-738b-4e92-9261-99055fa6ef93 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:58.288117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9749f68-3b8b-4700-8ba4-943f6f44d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:58.288865 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9749f68-3b8b-4700-8ba4-943f6f44d40f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.289144 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9749f68-3b8b-4700-8ba4-943f6f44d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.289416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9749f68-3b8b-4700-8ba4-943f6f44d40f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.290065 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9749f68-3b8b-4700-8ba4-943f6f44d40f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:58.291730 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9749f68-3b8b-4700-8ba4-943f6f44d40f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:58.305148 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1367/2730] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:11:58.309837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:58.321357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee721-38f3-426a-a3f8-8ef190fd4487 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:58.326424 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:58.327126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.327466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.327882 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] PATH_INFO: `/v3/projects/2b914a9f5c7a431d889feaf0e32a4a14/users/24b3c63d7d46473c817692d54b3a8cc8/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.350248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b914a9f5c7a431d889feaf0e32a4a14, user_id=24b3c63d7d46473c817692d54b3a8cc8, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:58.354617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:58.371740 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1364/2731] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:11:58 2022] PUT /identity/v3/projects/2b914a9f5c7a431d889feaf0e32a4a14/users/24b3c63d7d46473c817692d54b3a8cc8/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:11:58.383563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.383843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.384117 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.418948 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] MFA Rules not processed for user `24b3c63d7d46473c817692d54b3a8cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:11:58.420240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:58.497236 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1368/2732] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:11:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:11:58.509763 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:11:58.512045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:11:58.619597 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60b9f5-3c79-44a8-9ef6-ada2744784e9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:11:58.626026 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bddb3327380d43ef9b24277a3b308b0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:11:58.627223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:11:58.627774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:11:58.628252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:11:58.644347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:11:58.647818 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:11:58.660137 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1365/2733] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:11:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:01.296250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:01.297914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:01.331022 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:01.408830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f044b0-98f5-40af-93fa-de08078e2477 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:01.416368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:01.417242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:01.417710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:01.417990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] PATH_INFO: `/v3/users/9d73ba36238f449dbe08d8888920510f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:01.419243 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d73ba36238f449dbe08d8888920510f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:01.421832 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:01.494158 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d73ba36238f449dbe08d8888920510f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.509417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d73ba36238f449dbe08d8888920510f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.526515 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d73ba36238f449dbe08d8888920510f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.527370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d73ba36238f449dbe08d8888920510f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.527988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6367a4e3-7200-440d-8dde-12d9e0f73038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d73ba36238f449dbe08d8888920510f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.544711 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1369/2734] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:01 2022] DELETE /identity/v3/users/9d73ba36238f449dbe08d8888920510f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:01.736415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:01.757493 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b660b-ac88-4c5a-b2f9-6ee38b5dccca service glance] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:01.820347 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:01.821042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:01.821321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:01.821595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] PATH_INFO: `/v3/projects/b388fc6841ed4e5598fba4e8856b7af5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:01.822747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b388fc6841ed4e5598fba4e8856b7af5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:01.824680 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:01.848841 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:01.875211 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b388fc6841ed4e5598fba4e8856b7af5', 'request_id': 'req-bcc4f7e4-43f3-40a4-a299-807dba9e294e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.884829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b388fc6841ed4e5598fba4e8856b7af5', 'request_id': 'req-bcc4f7e4-43f3-40a4-a299-807dba9e294e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.890875 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b388fc6841ed4e5598fba4e8856b7af5', 'request_id': 'req-bcc4f7e4-43f3-40a4-a299-807dba9e294e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.902650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] The token cache is being invalidate because project b388fc6841ed4e5598fba4e8856b7af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:01.902937 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.903667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bcc4f7e4-43f3-40a4-a299-807dba9e294e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:01.905438 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1366/2735] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:01 2022] DELETE /identity/v3/projects/b388fc6841ed4e5598fba4e8856b7af5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:01.910993 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa475-62fa-4efc-b968-65fe9fb4ecab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:01.911313 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa475-62fa-4efc-b968-65fe9fb4ecab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:01.911535 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa475-62fa-4efc-b968-65fe9fb4ecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:01.931836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-446aa475-62fa-4efc-b968-65fe9fb4ecab None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:01.933137 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-446aa475-62fa-4efc-b968-65fe9fb4ecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:01.996222 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1370/2736] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:02.005138 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b364a-5019-49f8-a0a3-b894951847c1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.005476 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b364a-5019-49f8-a0a3-b894951847c1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.005757 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b364a-5019-49f8-a0a3-b894951847c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.026894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-887b364a-5019-49f8-a0a3-b894951847c1 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:02.028126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-887b364a-5019-49f8-a0a3-b894951847c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:02.036750 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1367/2737] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:02.041438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa475-62fa-4efc-b968-65fe9fb4ecab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.051966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa475-62fa-4efc-b968-65fe9fb4ecab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.056777 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.057385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.057665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.057937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.058580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:02.058881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.060519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.074180 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1371/2738] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:02.078571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b364a-5019-49f8-a0a3-b894951847c1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.088462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b364a-5019-49f8-a0a3-b894951847c1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.093225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.093973 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.094285 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.094582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.095236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.097249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.126459 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1368/2739] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:02 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:02.131747 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.142126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a1d5a-2359-4cf9-9c95-a56a00e9f98d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.166050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:02.217104 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.217751 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.218094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.218324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.218853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.220841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.245530 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1372/2740] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:02 2022] POST /identity/v3/users => generated 282 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:02.250016 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.262735 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf53852-6daa-466f-b4af-6fa164e7bb5e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.267798 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.268364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.268815 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.268985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.269635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.271321 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.283180 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1369/2741] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:02.288500 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.299464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f125e10-3f6b-4552-8f1e-7467bc00f86a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.305289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09721a46-af5d-4228-b09c-70508f7f4c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.306013 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721a46-af5d-4228-b09c-70508f7f4c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.306302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721a46-af5d-4228-b09c-70508f7f4c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.306575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721a46-af5d-4228-b09c-70508f7f4c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.307414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09721a46-af5d-4228-b09c-70508f7f4c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.309314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09721a46-af5d-4228-b09c-70508f7f4c63 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.322090 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1373/2742] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:02.326604 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.337388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1edb5-3a4f-47eb-b9f3-0e39102c0d7c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.345300 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.345987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.346267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.346538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] PATH_INFO: `/v3/projects/124e3ce1c6dc4d8c9ed543a85f25fb4c/users/6ee3be8fc34d4c56b8b13a40c1a55b2c/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.365957 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=124e3ce1c6dc4d8c9ed543a85f25fb4c, user_id=6ee3be8fc34d4c56b8b13a40c1a55b2c, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.368738 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.383554 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1370/2743] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:02 2022] PUT /identity/v3/projects/124e3ce1c6dc4d8c9ed543a85f25fb4c/users/6ee3be8fc34d4c56b8b13a40c1a55b2c/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:02.395332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a3e1-e3ce-45fa-b768-186f9166ab3e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.395574 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a3e1-e3ce-45fa-b768-186f9166ab3e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.395851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a3e1-e3ce-45fa-b768-186f9166ab3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.425674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cb25a3e1-e3ce-45fa-b768-186f9166ab3e None None] MFA Rules not processed for user `6ee3be8fc34d4c56b8b13a40c1a55b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:02.427408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cb25a3e1-e3ce-45fa-b768-186f9166ab3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:02.495168 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1374/2744] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:02.500111 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.565870 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73381087-3b0e-4298-98a5-bd92431632f9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.571380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.572014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.572297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.572605 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.573257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.575182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.612091 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1371/2745] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:02 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:02.613308 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25a3e1-e3ce-45fa-b768-186f9166ab3e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.629324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25a3e1-e3ce-45fa-b768-186f9166ab3e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.690752 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.691368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.691652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.691928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.692682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.694636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.717988 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1375/2746] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:02 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:02.722050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.734490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65bb4-46ce-464e-b48f-cf189a21a763 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.739935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.740643 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.740911 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.741186 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.741828 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.743419 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.755963 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1372/2747] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:02.760640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.770676 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d82a24-cd09-494a-90d7-01d32cbd99be admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.775526 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.776185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.776496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.776771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] PATH_INFO: `/v3/projects/796d9f213da14630af2714232354564b/users/e9711a80d2b44006aeaf6735b582e2b1/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.793891 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796d9f213da14630af2714232354564b, user_id=e9711a80d2b44006aeaf6735b582e2b1, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.796582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.799671 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:02.812616 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1376/2748] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:02 2022] PUT /identity/v3/projects/796d9f213da14630af2714232354564b/users/e9711a80d2b44006aeaf6735b582e2b1/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:02.816824 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.827135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076852f5-0104-475a-ad82-43f28ba4ff11 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.840951 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:02.890328 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bec5fe-6951-42d7-a50f-2957d9fcf21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.890942 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec5fe-6951-42d7-a50f-2957d9fcf21b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.891250 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec5fe-6951-42d7-a50f-2957d9fcf21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.891530 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec5fe-6951-42d7-a50f-2957d9fcf21b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.892175 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bec5fe-6951-42d7-a50f-2957d9fcf21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.893829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bec5fe-6951-42d7-a50f-2957d9fcf21b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.905297 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1373/2749] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:02.910458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:02.923329 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f0f2d-b317-4d76-868d-24019620dec2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:02.928149 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:02.928978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.929299 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.929610 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] PATH_INFO: `/v3/projects/796d9f213da14630af2714232354564b/users/e9711a80d2b44006aeaf6735b582e2b1/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.931722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=796d9f213da14630af2714232354564b, user_id=e9711a80d2b44006aeaf6735b582e2b1, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:02.934551 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:02.945623 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1377/2750] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:02 2022] PUT /identity/v3/projects/796d9f213da14630af2714232354564b/users/e9711a80d2b44006aeaf6735b582e2b1/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:02.955941 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd11e-7fc0-4913-9fa9-d8c226b219ed None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:02.956228 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd11e-7fc0-4913-9fa9-d8c226b219ed None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:02.956543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bcd11e-7fc0-4913-9fa9-d8c226b219ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:02.988096 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a1bcd11e-7fc0-4913-9fa9-d8c226b219ed None None] MFA Rules not processed for user `e9711a80d2b44006aeaf6735b582e2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:02.989374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a1bcd11e-7fc0-4913-9fa9-d8c226b219ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:03.069370 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1374/2751] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:03.081109 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:03.083421 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:03.143352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fe158-2886-406b-a7be-168c7b9bc01a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:03.147761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd250e74-797d-496f-a9c6-0b97749b8c70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:03.148399 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd250e74-797d-496f-a9c6-0b97749b8c70 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:03.148711 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd250e74-797d-496f-a9c6-0b97749b8c70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:03.148963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd250e74-797d-496f-a9c6-0b97749b8c70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:03.158172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd250e74-797d-496f-a9c6-0b97749b8c70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:03.159622 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd250e74-797d-496f-a9c6-0b97749b8c70 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:03.170633 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1378/2752] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:03.637573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcd11e-7fc0-4913-9fa9-d8c226b219ed None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:03.701881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bcd11e-7fc0-4913-9fa9-d8c226b219ed None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:03.706494 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:03.707166 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:03.707448 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:03.707730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] PATH_INFO: `/v3/users/6ee3be8fc34d4c56b8b13a40c1a55b2c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:03.708922 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ee3be8fc34d4c56b8b13a40c1a55b2c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:03.710985 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:03.756627 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:03.763748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ee3be8fc34d4c56b8b13a40c1a55b2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:03.764988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ee3be8fc34d4c56b8b13a40c1a55b2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:03.773178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ee3be8fc34d4c56b8b13a40c1a55b2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:03.785387 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ee3be8fc34d4c56b8b13a40c1a55b2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:03.786479 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ee3be8fc34d4c56b8b13a40c1a55b2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:03.802768 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1375/2753] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:03 2022] DELETE /identity/v3/users/6ee3be8fc34d4c56b8b13a40c1a55b2c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:03.807582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:03.808083 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:03.808214 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:03.828990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:03.830206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:03.894101 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1379/2754] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:03.905079 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:03.906719 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:03.969884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e617d2-1f4a-4d2f-9b04-9e61c11fcb16 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:03.975064 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:03.975928 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:03.976257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:03.976645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:03.988244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:03.990366 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:04.000610 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1376/2755] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:04.294410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:04.295979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:04.316523 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350adf23-8f83-4b3b-bbc3-285c7498fca5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:04.321640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:04.322266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:04.322627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:04.322908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] PATH_INFO: `/v3/projects/124e3ce1c6dc4d8c9ed543a85f25fb4c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:04.324113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=124e3ce1c6dc4d8c9ed543a85f25fb4c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:04.326463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:04.381582 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124e3ce1c6dc4d8c9ed543a85f25fb4c', 'request_id': 'req-e9237854-fd60-406f-bb98-150b84d978d7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.388125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124e3ce1c6dc4d8c9ed543a85f25fb4c', 'request_id': 'req-e9237854-fd60-406f-bb98-150b84d978d7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.394089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124e3ce1c6dc4d8c9ed543a85f25fb4c', 'request_id': 'req-e9237854-fd60-406f-bb98-150b84d978d7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.409050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] The token cache is being invalidate because project 124e3ce1c6dc4d8c9ed543a85f25fb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:04.409316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.409896 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.412299 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1380/2756] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:04 2022] DELETE /identity/v3/projects/124e3ce1c6dc4d8c9ed543a85f25fb4c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:04.416401 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:04.426367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f22195-2f6f-4d3f-aaed-e12b6867ab83 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:04.488837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:04.489489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:04.489759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:04.490054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] PATH_INFO: `/v3/users/e9711a80d2b44006aeaf6735b582e2b1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:04.491185 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9711a80d2b44006aeaf6735b582e2b1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:04.493124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:04.547345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9711a80d2b44006aeaf6735b582e2b1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.548023 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9711a80d2b44006aeaf6735b582e2b1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.556204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9711a80d2b44006aeaf6735b582e2b1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.562398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:04.571299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9711a80d2b44006aeaf6735b582e2b1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.572103 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d9c39-6c54-4090-b3de-d49cab0acadd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9711a80d2b44006aeaf6735b582e2b1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:04.597113 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1377/2757] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:04 2022] DELETE /identity/v3/users/e9711a80d2b44006aeaf6735b582e2b1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:04.907666 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:04.909553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:04.927020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9237854-fd60-406f-bb98-150b84d978d7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:04.932072 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:04.993460 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:04.994070 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:04.994398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:04.994703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] PATH_INFO: `/v3/projects/796d9f213da14630af2714232354564b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:04.995970 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=796d9f213da14630af2714232354564b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:04.998238 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.055903 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796d9f213da14630af2714232354564b', 'request_id': 'req-ad0d82c2-45e8-4a03-900f-32ab44564735'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:05.062496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796d9f213da14630af2714232354564b', 'request_id': 'req-ad0d82c2-45e8-4a03-900f-32ab44564735'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:05.069341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796d9f213da14630af2714232354564b', 'request_id': 'req-ad0d82c2-45e8-4a03-900f-32ab44564735'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:05.083998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] The token cache is being invalidate because project 796d9f213da14630af2714232354564b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:05.084287 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:05.085052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ad0d82c2-45e8-4a03-900f-32ab44564735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:05.086742 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1381/2758] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:04 2022] DELETE /identity/v3/projects/796d9f213da14630af2714232354564b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:05.093888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36888cd4-2b49-4803-b457-92304b1e5b8c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.093888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36888cd4-2b49-4803-b457-92304b1e5b8c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.093888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36888cd4-2b49-4803-b457-92304b1e5b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.123743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-36888cd4-2b49-4803-b457-92304b1e5b8c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:05.123743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-36888cd4-2b49-4803-b457-92304b1e5b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:05.188453 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1378/2759] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:05.197283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780bdae-fdbb-4541-9d67-5ecbc9146965 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.197580 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780bdae-fdbb-4541-9d67-5ecbc9146965 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.197866 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780bdae-fdbb-4541-9d67-5ecbc9146965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.220055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c780bdae-fdbb-4541-9d67-5ecbc9146965 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:05.221311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c780bdae-fdbb-4541-9d67-5ecbc9146965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:05.230138 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1382/2760] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:05.234670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36888cd4-2b49-4803-b457-92304b1e5b8c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.245176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36888cd4-2b49-4803-b457-92304b1e5b8c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.249703 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.250288 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.250589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.250861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.251499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:05.251804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.253668 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.271364 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1379/2761] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:05.272753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780bdae-fdbb-4541-9d67-5ecbc9146965 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.282418 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780bdae-fdbb-4541-9d67-5ecbc9146965 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.287564 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.288179 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.288490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.288765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.289428 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.291348 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.325437 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1383/2762] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:05 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:05.330010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.343325 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77fbca-a818-4dcd-955b-76828cca0c7e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.404966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.405582 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.405872 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.406149 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.406827 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.409223 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.434139 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1380/2763] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:05 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:05.438461 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.450173 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55607a-3ce3-471c-a090-b6d9c4041c16 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.455761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.456411 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.456696 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.456963 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.457661 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.459275 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.470117 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1384/2764] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:05.474803 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.487561 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fef13-1a22-4dcf-9165-5b50473d5ef3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.493045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4477cc1-3c1e-4824-a644-797bdbf17ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.493659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4477cc1-3c1e-4824-a644-797bdbf17ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.493933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4477cc1-3c1e-4824-a644-797bdbf17ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.494938 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4477cc1-3c1e-4824-a644-797bdbf17ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.495662 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4477cc1-3c1e-4824-a644-797bdbf17ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.497225 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4477cc1-3c1e-4824-a644-797bdbf17ff5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.510820 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1381/2765] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:05.515553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.526734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea8e79-1da2-4658-82d2-a53d65ceab9d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.538051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.538051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.538051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.538051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] PATH_INFO: `/v3/projects/6f89ceafdc234f66a869a766ae93da2b/users/74a2560fec7e4b8da5e82eef7d70a146/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.555851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f89ceafdc234f66a869a766ae93da2b, user_id=74a2560fec7e4b8da5e82eef7d70a146, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.559268 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.575231 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1385/2766] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:05 2022] PUT /identity/v3/projects/6f89ceafdc234f66a869a766ae93da2b/users/74a2560fec7e4b8da5e82eef7d70a146/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:05.587560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fe84d-ee77-4852-86ac-0d5c9ee9a372 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.587844 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fe84d-ee77-4852-86ac-0d5c9ee9a372 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.588244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fe84d-ee77-4852-86ac-0d5c9ee9a372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.618416 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eb7fe84d-ee77-4852-86ac-0d5c9ee9a372 None None] MFA Rules not processed for user `74a2560fec7e4b8da5e82eef7d70a146`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:05.619682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eb7fe84d-ee77-4852-86ac-0d5c9ee9a372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:05.655348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:05.689205 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1382/2767] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:05.694506 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.736490 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:05.764971 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fe645-eb94-4652-b848-49194558a04f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.769616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.770849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.771154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.771425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.772067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.774229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.804362 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1386/2768] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:05 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:05.809161 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fe84d-ee77-4852-86ac-0d5c9ee9a372 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.821417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fe84d-ee77-4852-86ac-0d5c9ee9a372 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.886011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.886637 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.886923 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.887197 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.887881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.889987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.920221 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1383/2769] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:05 2022] POST /identity/v3/users => generated 301 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:05.921706 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.934103 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1630b9-8ec3-4ad8-8b2c-9378c2cfb936 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.941337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.942002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.942290 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.942571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:05.943253 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:05.945002 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:05.956517 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1387/2770] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:05.961596 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:05.974496 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b049b1-6993-4891-8581-78540bb808ad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:05.979774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:05.980620 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:05.980987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:05.981380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] PATH_INFO: `/v3/projects/e8c8746040ff418b861267ac94ea07c3/users/e29316e0f6be4cb1aab4e43d7dc42248/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:06.000776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c8746040ff418b861267ac94ea07c3, user_id=e29316e0f6be4cb1aab4e43d7dc42248, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:06.003799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:06.024218 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1384/2771] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:05 2022] PUT /identity/v3/projects/e8c8746040ff418b861267ac94ea07c3/users/e29316e0f6be4cb1aab4e43d7dc42248/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:06.028234 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:06.040769 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984af1e5-9b27-408f-b7ae-06a12e276307 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:06.103984 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593d4a6-4e48-4fa5-82c8-1e9533576c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:06.104642 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593d4a6-4e48-4fa5-82c8-1e9533576c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:06.104926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593d4a6-4e48-4fa5-82c8-1e9533576c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:06.105201 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593d4a6-4e48-4fa5-82c8-1e9533576c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:06.105850 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593d4a6-4e48-4fa5-82c8-1e9533576c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:06.107459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593d4a6-4e48-4fa5-82c8-1e9533576c3f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:06.118002 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1388/2772] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:06.122433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:06.133682 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f10d6-a3bf-438a-8ff5-3327f2c697bc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:06.138247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:06.138904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:06.139204 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:06.139475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] PATH_INFO: `/v3/projects/e8c8746040ff418b861267ac94ea07c3/users/e29316e0f6be4cb1aab4e43d7dc42248/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:06.141396 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c8746040ff418b861267ac94ea07c3, user_id=e29316e0f6be4cb1aab4e43d7dc42248, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:06.143998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:06.155695 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1385/2773] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:06 2022] PUT /identity/v3/projects/e8c8746040ff418b861267ac94ea07c3/users/e29316e0f6be4cb1aab4e43d7dc42248/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:06.167181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:06.167471 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:06.167750 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:06.198286 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] MFA Rules not processed for user `e29316e0f6be4cb1aab4e43d7dc42248`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:06.199499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:06.271755 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1389/2774] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:06 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:06.282481 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:06.284222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:06.348691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c837b38-e963-4142-a3ff-3a5ebe5e2199 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:06.352373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:06.353010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:06.353294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:06.353568 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:06.363195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:06.365310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:06.375012 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1386/2775] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:06 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:06.424498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:06.426232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:06.438588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc93b-0253-43cf-9b9b-e863111cd60c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:06.442880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c27225-3615-4994-bfb9-cf8f58904ad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:06.443478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c27225-3615-4994-bfb9-cf8f58904ad1 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:06.443754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c27225-3615-4994-bfb9-cf8f58904ad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:06.444023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c27225-3615-4994-bfb9-cf8f58904ad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:06.479417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:06.516803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c27225-3615-4994-bfb9-cf8f58904ad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:06.518301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c27225-3615-4994-bfb9-cf8f58904ad1 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:06.530523 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1390/2776] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:09.924443 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:09.964567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:09.998296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232f174-9850-4238-83e3-d06aae703b90 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:10.003783 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:10.005019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:10.005297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:10.005572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] PATH_INFO: `/v3/users/74a2560fec7e4b8da5e82eef7d70a146` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:10.007656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a2560fec7e4b8da5e82eef7d70a146)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:10.009217 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:10.064665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a2560fec7e4b8da5e82eef7d70a146'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.065362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a2560fec7e4b8da5e82eef7d70a146'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.073128 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a2560fec7e4b8da5e82eef7d70a146'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.087249 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a2560fec7e4b8da5e82eef7d70a146'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.088014 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a2560fec7e4b8da5e82eef7d70a146'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.102214 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1387/2777] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:09 2022] DELETE /identity/v3/users/74a2560fec7e4b8da5e82eef7d70a146 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:10.107023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:10.107545 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:10.107672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:10.127887 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:10.129163 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:10.195721 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1391/2778] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:10.206368 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:10.208453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:10.265095 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47860b88-251e-4828-b847-7bc1b0b5940d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:10.268755 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:10.269360 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:10.269639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:10.270619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:10.279667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:10.281126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:10.291664 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1388/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:10.581618 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:10.583371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:10.596098 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3df22-74f0-4fcc-825d-0325b2412bba None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:10.601922 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:10.602946 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:10.603227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:10.603524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] PATH_INFO: `/v3/projects/6f89ceafdc234f66a869a766ae93da2b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:10.605100 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f89ceafdc234f66a869a766ae93da2b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:10.607216 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:10.661628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f89ceafdc234f66a869a766ae93da2b', 'request_id': 'req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.668097 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f89ceafdc234f66a869a766ae93da2b', 'request_id': 'req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.673908 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f89ceafdc234f66a869a766ae93da2b', 'request_id': 'req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.687059 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] The token cache is being invalidate because project 6f89ceafdc234f66a869a766ae93da2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:10.687353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.687989 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.689511 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1392/2780] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:10 2022] DELETE /identity/v3/projects/6f89ceafdc234f66a869a766ae93da2b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:10.694772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:10.705143 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffc1fa-0bef-4068-a0b5-9436db5ea57e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:10.753864 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:10.764898 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:10.765532 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:10.765839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:10.766114 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] PATH_INFO: `/v3/users/e29316e0f6be4cb1aab4e43d7dc42248` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:10.767291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e29316e0f6be4cb1aab4e43d7dc42248)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:10.769247 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:10.819167 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e29316e0f6be4cb1aab4e43d7dc42248'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.819956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e29316e0f6be4cb1aab4e43d7dc42248'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.828117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e29316e0f6be4cb1aab4e43d7dc42248'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.841881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e29316e0f6be4cb1aab4e43d7dc42248'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.842672 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3d9144a9-ee91-47e3-b625-e682c2cc740d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e29316e0f6be4cb1aab4e43d7dc42248'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:10.857829 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1389/2781] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:10 2022] DELETE /identity/v3/users/e29316e0f6be4cb1aab4e43d7dc42248 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:11.147404 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:11.149115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:11.167994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb2b96-8db4-4a32-8d94-e5c9e8cda177 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:11.220454 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:11.231324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:11.231962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.232233 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.232534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] PATH_INFO: `/v3/projects/e8c8746040ff418b861267ac94ea07c3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.233681 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c8746040ff418b861267ac94ea07c3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:11.235576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:11.282763 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c8746040ff418b861267ac94ea07c3', 'request_id': 'req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:11.290003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c8746040ff418b861267ac94ea07c3', 'request_id': 'req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:11.295781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c8746040ff418b861267ac94ea07c3', 'request_id': 'req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:11.309628 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] The token cache is being invalidate because project e8c8746040ff418b861267ac94ea07c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:11.309914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:11.310747 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6f4c5c1c-d76f-42c5-8a85-56d2f79663a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:11.312190 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1393/2782] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:11 2022] DELETE /identity/v3/projects/e8c8746040ff418b861267ac94ea07c3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:11.317241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27464d08-69c3-4cea-bdfb-4c6fdf67ea85 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.317739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27464d08-69c3-4cea-bdfb-4c6fdf67ea85 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.317865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27464d08-69c3-4cea-bdfb-4c6fdf67ea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.339648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-27464d08-69c3-4cea-bdfb-4c6fdf67ea85 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:11.340913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-27464d08-69c3-4cea-bdfb-4c6fdf67ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:11.415014 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1390/2783] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:11.423228 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7149b-3a4e-45e9-8de1-1bd5c06554f4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.423508 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7149b-3a4e-45e9-8de1-1bd5c06554f4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.423779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7149b-3a4e-45e9-8de1-1bd5c06554f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.444521 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-73b7149b-3a4e-45e9-8de1-1bd5c06554f4 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:11.445742 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-73b7149b-3a4e-45e9-8de1-1bd5c06554f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:11.453859 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1394/2784] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:11.459020 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27464d08-69c3-4cea-bdfb-4c6fdf67ea85 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:11.470665 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27464d08-69c3-4cea-bdfb-4c6fdf67ea85 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:11.475482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:11.476099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.476538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.476688 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.477354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:11.477659 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:11.479280 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:11.492040 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1391/2785] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:11.496562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7149b-3a4e-45e9-8de1-1bd5c06554f4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:11.506649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7149b-3a4e-45e9-8de1-1bd5c06554f4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:11.512193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:11.513112 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.513398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.513674 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.514651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:11.517592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:11.550475 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1395/2786] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:11 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:11.555865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:11.568066 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74528094-ea75-416c-aaf8-7f70402f1cff admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:11.636043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:11.636702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.636982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.637255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.637906 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:11.639865 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:11.649332 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:11.670639 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1392/2787] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:11 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:11.675142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:11.689092 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c5ff9-df22-4b47-a790-a675d255bb15 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:11.697645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:11.698719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.699213 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.699707 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.700662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:11.702889 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:11.714389 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1396/2788] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:11.718664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:11.729532 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680820d7-f07d-463d-b638-0964f44bdca6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:11.734391 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872976b-714b-4840-bfae-b1c268a39204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:11.735002 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872976b-714b-4840-bfae-b1c268a39204 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.735306 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872976b-714b-4840-bfae-b1c268a39204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.735579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872976b-714b-4840-bfae-b1c268a39204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.736221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872976b-714b-4840-bfae-b1c268a39204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:11.737981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872976b-714b-4840-bfae-b1c268a39204 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:11.750084 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1393/2789] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:11.755931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:11.771708 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba7623-43ee-4c79-8874-8f78e9a8ca5d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:11.776885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:11.777592 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.777873 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.778192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] PATH_INFO: `/v3/projects/8848585fbceb4d37a30e70042839c946/users/dfda7e40db3540a485f2897204527719/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.795288 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8848585fbceb4d37a30e70042839c946, user_id=dfda7e40db3540a485f2897204527719, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:11.798016 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:11.814929 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1397/2790] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:11 2022] PUT /identity/v3/projects/8848585fbceb4d37a30e70042839c946/users/dfda7e40db3540a485f2897204527719/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:11.826913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b430f3-0bd6-446e-98dc-ab9a9361f648 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:11.827198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b430f3-0bd6-446e-98dc-ab9a9361f648 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:11.827475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b430f3-0bd6-446e-98dc-ab9a9361f648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:11.864264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e1b430f3-0bd6-446e-98dc-ab9a9361f648 None None] MFA Rules not processed for user `dfda7e40db3540a485f2897204527719`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:11.865490 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e1b430f3-0bd6-446e-98dc-ab9a9361f648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:11.937298 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1394/2791] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:11.944082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.016451 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72260d2-fbe9-4d5c-8df8-756a31027514 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.021162 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.021766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.022046 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.022320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.022988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.024928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.056085 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1398/2792] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:11 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:12.057244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b430f3-0bd6-446e-98dc-ab9a9361f648 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.068649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b430f3-0bd6-446e-98dc-ab9a9361f648 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.126105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.126714 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.126987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.127255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.127889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.130409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.151801 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1395/2793] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:12 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:12.155977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.163251 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:12.167266 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b24b6-242b-42dd-8194-923f4383205a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.173259 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.173941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.174355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.174641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.175464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.178058 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.188979 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1399/2794] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:12.193885 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.204063 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434db6ad-87b9-415d-8eef-4750ad5f9b4e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.209198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.209873 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.210159 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.210435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] PATH_INFO: `/v3/projects/4e5d8e9fb3b14060917c77dda511afe4/users/495215e48f53461c988b6b3cff77eb56/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.227074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5d8e9fb3b14060917c77dda511afe4, user_id=495215e48f53461c988b6b3cff77eb56, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.229804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.243410 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1396/2795] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:12 2022] PUT /identity/v3/projects/4e5d8e9fb3b14060917c77dda511afe4/users/495215e48f53461c988b6b3cff77eb56/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:12.247366 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.257835 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae232e-354c-4894-bcb3-9b2889a74e4b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.319256 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f8bd0-ff63-4636-b418-dd53853754f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.319802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8bd0-ff63-4636-b418-dd53853754f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.320082 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8bd0-ff63-4636-b418-dd53853754f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.320353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8bd0-ff63-4636-b418-dd53853754f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.321019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f8bd0-ff63-4636-b418-dd53853754f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.322616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f8bd0-ff63-4636-b418-dd53853754f8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.332972 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1400/2796] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:12.337176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.353843 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8a299-7714-4e2d-a29d-915ffcac3794 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.360267 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.361415 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.361686 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.361982 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] PATH_INFO: `/v3/projects/4e5d8e9fb3b14060917c77dda511afe4/users/495215e48f53461c988b6b3cff77eb56/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.366697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5d8e9fb3b14060917c77dda511afe4, user_id=495215e48f53461c988b6b3cff77eb56, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.372048 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.383218 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1397/2797] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:12 2022] PUT /identity/v3/projects/4e5d8e9fb3b14060917c77dda511afe4/users/495215e48f53461c988b6b3cff77eb56/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:12.394578 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ceae6-fad8-4301-83ca-f432e68a4be0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.395095 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ceae6-fad8-4301-83ca-f432e68a4be0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.395559 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ceae6-fad8-4301-83ca-f432e68a4be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.429542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9d1ceae6-fad8-4301-83ca-f432e68a4be0 None None] MFA Rules not processed for user `495215e48f53461c988b6b3cff77eb56`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:12.430747 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d1ceae6-fad8-4301-83ca-f432e68a4be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:12.500720 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1401/2798] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:12.511340 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.513093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:12.537080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:12.574570 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0925885-246a-4605-9d1e-0f4d4916f3a5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.578144 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.578749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.579178 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.579298 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.589749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.591148 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.601364 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1398/2799] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:12.712949 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ceae6-fad8-4301-83ca-f432e68a4be0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:12.724317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ceae6-fad8-4301-83ca-f432e68a4be0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:12.727985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1852bb-5ecd-47b0-bd22-0a59da5737ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:12.728626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1852bb-5ecd-47b0-bd22-0a59da5737ab service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:12.728906 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1852bb-5ecd-47b0-bd22-0a59da5737ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:12.729177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1852bb-5ecd-47b0-bd22-0a59da5737ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:12.786368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1852bb-5ecd-47b0-bd22-0a59da5737ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:12.787824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1852bb-5ecd-47b0-bd22-0a59da5737ab service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:12.796925 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1402/2800] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:15.434775 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:15.515115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c5aa2-66cb-40f1-b8ef-e98fc17b0aee service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:15.520666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:15.521315 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:15.521648 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:15.521920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] PATH_INFO: `/v3/users/dfda7e40db3540a485f2897204527719` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:15.523461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfda7e40db3540a485f2897204527719)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:15.525573 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:15.578826 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfda7e40db3540a485f2897204527719'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:15.579651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfda7e40db3540a485f2897204527719'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:15.590371 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfda7e40db3540a485f2897204527719'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:15.603711 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfda7e40db3540a485f2897204527719'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:15.604500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfda7e40db3540a485f2897204527719'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:15.620539 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1399/2801] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:15 2022] DELETE /identity/v3/users/dfda7e40db3540a485f2897204527719 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:15.625380 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:15.626641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:15.626926 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:15.648106 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:15.649374 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:15.658099 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:15.719979 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1403/2802] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:15.730447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:15.731947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:15.781275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:15.796275 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee0e27-eeea-4a30-b6ac-e3e05234dc17 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:15.800068 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:15.800721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:15.801612 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:15.801891 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:15.811737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:15.813215 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:15.824884 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1400/2803] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:16.135710 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:16.138037 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:16.165020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e899998-c551-4d90-9fc7-4d0553804a2a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:16.173241 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:16.174281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:16.174760 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:16.175335 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] PATH_INFO: `/v3/projects/8848585fbceb4d37a30e70042839c946` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:16.177197 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8848585fbceb4d37a30e70042839c946)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:16.179252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:16.239561 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8848585fbceb4d37a30e70042839c946', 'request_id': 'req-32c62400-702a-424b-a981-dfcb8dfe72e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.246755 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8848585fbceb4d37a30e70042839c946', 'request_id': 'req-32c62400-702a-424b-a981-dfcb8dfe72e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.253868 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8848585fbceb4d37a30e70042839c946', 'request_id': 'req-32c62400-702a-424b-a981-dfcb8dfe72e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.271557 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] The token cache is being invalidate because project 8848585fbceb4d37a30e70042839c946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:16.271969 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.272888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.274927 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1404/2804] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:16 2022] DELETE /identity/v3/projects/8848585fbceb4d37a30e70042839c946 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:16.281348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:16.293647 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c795fce-95b9-4650-ac4d-3579c3d32247 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:16.510354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:16.510996 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:16.511282 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:16.511554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] PATH_INFO: `/v3/users/495215e48f53461c988b6b3cff77eb56` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:16.512778 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=495215e48f53461c988b6b3cff77eb56)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:16.514759 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:16.581674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495215e48f53461c988b6b3cff77eb56'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.582543 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495215e48f53461c988b6b3cff77eb56'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.593132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495215e48f53461c988b6b3cff77eb56'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.618195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495215e48f53461c988b6b3cff77eb56'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.619264 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5a7632f5-b1f7-43bf-815e-8084f6961dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495215e48f53461c988b6b3cff77eb56'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:16.642017 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1401/2805] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:16 2022] DELETE /identity/v3/users/495215e48f53461c988b6b3cff77eb56 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:16.966370 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:16.967867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:16.984975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62400-702a-424b-a981-dfcb8dfe72e3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.069977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.070829 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.071344 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.071652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] PATH_INFO: `/v3/projects/4e5d8e9fb3b14060917c77dda511afe4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.072846 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e5d8e9fb3b14060917c77dda511afe4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.074753 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.129289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:17.139463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e5d8e9fb3b14060917c77dda511afe4', 'request_id': 'req-c6c9211d-bdca-446a-ab06-57249c728c07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.147442 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e5d8e9fb3b14060917c77dda511afe4', 'request_id': 'req-c6c9211d-bdca-446a-ab06-57249c728c07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.153463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e5d8e9fb3b14060917c77dda511afe4', 'request_id': 'req-c6c9211d-bdca-446a-ab06-57249c728c07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.170438 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] The token cache is being invalidate because project 4e5d8e9fb3b14060917c77dda511afe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:17.170806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.171491 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6c9211d-bdca-446a-ab06-57249c728c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.173319 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1405/2806] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:16 2022] DELETE /identity/v3/projects/4e5d8e9fb3b14060917c77dda511afe4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:17.180038 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bdca8-697e-4b85-b471-038727eece36 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.180356 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bdca8-697e-4b85-b471-038727eece36 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.180660 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bdca8-697e-4b85-b471-038727eece36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.200796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0c3bdca8-697e-4b85-b471-038727eece36 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:17.201998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0c3bdca8-697e-4b85-b471-038727eece36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:17.265629 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1402/2807] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:17.275202 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a34e45-b14f-4cbe-9ec3-5f6d0223196b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.275487 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a34e45-b14f-4cbe-9ec3-5f6d0223196b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.275767 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a34e45-b14f-4cbe-9ec3-5f6d0223196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.297613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-78a34e45-b14f-4cbe-9ec3-5f6d0223196b None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:17.298848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-78a34e45-b14f-4cbe-9ec3-5f6d0223196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:17.307203 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1406/2808] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:17.311840 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bdca8-697e-4b85-b471-038727eece36 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.318447 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:17.322549 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bdca8-697e-4b85-b471-038727eece36 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.329154 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.329787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.330276 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.330771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.331635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:17.331999 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.334165 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.346499 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1403/2809] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:17.351729 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a34e45-b14f-4cbe-9ec3-5f6d0223196b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.361766 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a34e45-b14f-4cbe-9ec3-5f6d0223196b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.366820 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.367430 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.367724 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.367990 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.368665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.370614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.399043 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1407/2810] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:17.406712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.419024 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120c2c6-b953-4eed-a25a-bee8d81c8070 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.476614 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.477281 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.477613 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.477935 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.478609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.480589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.502569 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1404/2811] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:17.507312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.518573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5beb6c-b541-44d6-9f36-886b9f36dc8a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.524685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.525331 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.525613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.525888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.526541 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.528147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.538819 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1408/2812] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:17.543054 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.553244 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c659b-f5f8-4a87-8985-eda02b73bdd5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.557901 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ae189-6025-4929-ac84-1d6e96e44aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.558501 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ae189-6025-4929-ac84-1d6e96e44aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.558780 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ae189-6025-4929-ac84-1d6e96e44aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.559052 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ae189-6025-4929-ac84-1d6e96e44aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.559722 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ae189-6025-4929-ac84-1d6e96e44aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.561294 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ae189-6025-4929-ac84-1d6e96e44aeb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.572043 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1405/2813] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:17.576431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.587297 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32290-c3ee-4d74-96fc-912615d293e8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.592229 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.592914 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.593192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.593464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] PATH_INFO: `/v3/projects/b1579c5a00e749999cf06f0551913b81/users/937f426b7053489c85cb50d92c3c358d/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.609596 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1579c5a00e749999cf06f0551913b81, user_id=937f426b7053489c85cb50d92c3c358d, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.612311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.625861 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1409/2814] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:17 2022] PUT /identity/v3/projects/b1579c5a00e749999cf06f0551913b81/users/937f426b7053489c85cb50d92c3c358d/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:17.636691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04481b66-7c5e-492b-8633-952045e126fe None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.636979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04481b66-7c5e-492b-8633-952045e126fe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.637236 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04481b66-7c5e-492b-8633-952045e126fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.665770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-04481b66-7c5e-492b-8633-952045e126fe None None] MFA Rules not processed for user `937f426b7053489c85cb50d92c3c358d`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:17.666987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-04481b66-7c5e-492b-8633-952045e126fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:17.691555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.693431 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:17.736997 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1406/2815] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:17.741987 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04481b66-7c5e-492b-8633-952045e126fe None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.761291 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab3ae8-1ec2-4c98-bad5-6d1a1a80c142 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.766759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.767400 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.767685 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.767965 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] PATH_INFO: `/v3/users/24b3c63d7d46473c817692d54b3a8cc8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.769150 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b3c63d7d46473c817692d54b3a8cc8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.771164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.805465 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04481b66-7c5e-492b-8633-952045e126fe None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.811045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.811661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.811971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.812242 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.812921 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.814860 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:17.819664 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:17.821426 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b3c63d7d46473c817692d54b3a8cc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.829830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b3c63d7d46473c817692d54b3a8cc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.839609 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:17.845723 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b3c63d7d46473c817692d54b3a8cc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.846987 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b3c63d7d46473c817692d54b3a8cc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.847636 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-901cbea1-3770-4dce-bf87-0dc6d5d70f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b3c63d7d46473c817692d54b3a8cc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:17.849652 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1407/2816] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:17.853823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.855489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:17.867620 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1410/2817] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:17 2022] DELETE /identity/v3/users/24b3c63d7d46473c817692d54b3a8cc8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:17.871947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.872232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.872542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.891569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:17.892852 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:17.924489 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aaf14b-5a8a-4e89-831c-519c08cf122a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:17.956118 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1411/2818] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:17.966967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:17.969038 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:17.986398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:17.987018 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:17.987293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:17.987567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:17.988193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:17.990146 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.012280 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1408/2819] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:17 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:18.016139 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.026776 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f347c-905e-48ed-b8a2-85097429ecf2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.028474 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c2a99-3514-45a7-9f6d-0b7c160cdaba None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.031577 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.032179 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.032302 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.032484 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.032772 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.032880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.033178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.033448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.033572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.035057 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.043565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.045052 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.047331 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1409/2820] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:12:18.052537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.055991 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1412/2821] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:12:18.062743 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84456ab0-31a0-4c08-894d-bfeb1c115c03 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.067560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.068260 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.068554 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.068833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] PATH_INFO: `/v3/projects/2a37180c429041e6992e30607ab811c3/users/a945e53b86614b079efdef3025080eae/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.085201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a37180c429041e6992e30607ab811c3, user_id=a945e53b86614b079efdef3025080eae, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.092182 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.111470 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1410/2822] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:18 2022] PUT /identity/v3/projects/2a37180c429041e6992e30607ab811c3/users/a945e53b86614b079efdef3025080eae/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:18.115657 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.127670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465870-6100-44fb-b8d4-7821f2231ca2 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.190626 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7965c1-4068-48b7-b3cc-a02c855c6d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.192332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7965c1-4068-48b7-b3cc-a02c855c6d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.192530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7965c1-4068-48b7-b3cc-a02c855c6d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.192814 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7965c1-4068-48b7-b3cc-a02c855c6d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.193543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7965c1-4068-48b7-b3cc-a02c855c6d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.195148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7965c1-4068-48b7-b3cc-a02c855c6d5d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.208573 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1413/2823] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:18.213081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.224436 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6912d57-c547-4822-a8e3-3e6e8eb6feb8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.229257 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.229992 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.230326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.230602 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] PATH_INFO: `/v3/projects/2a37180c429041e6992e30607ab811c3/users/a945e53b86614b079efdef3025080eae/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.232548 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a37180c429041e6992e30607ab811c3, user_id=a945e53b86614b079efdef3025080eae, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.235235 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.247729 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1411/2824] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:18 2022] PUT /identity/v3/projects/2a37180c429041e6992e30607ab811c3/users/a945e53b86614b079efdef3025080eae/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:18.259174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.259542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.259870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.292964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] MFA Rules not processed for user `a945e53b86614b079efdef3025080eae`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:18.294503 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:18.311388 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:18.343658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.343658 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:18.386792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:18.391088 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1414/2825] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:18.402871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.405086 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:18.424685 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c253da3-3c7c-4728-8883-f052a3a8f8cd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.430301 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.430930 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.431208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.431482 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] PATH_INFO: `/v3/projects/2b914a9f5c7a431d889feaf0e32a4a14` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.432639 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b914a9f5c7a431d889feaf0e32a4a14)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.434534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.465562 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a49ae-aded-4e09-a721-11e162b9859e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.470364 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d419b1-e5b4-4150-a014-b285a4f6f3f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.471110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d419b1-e5b4-4150-a014-b285a4f6f3f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.471391 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d419b1-e5b4-4150-a014-b285a4f6f3f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.471662 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d419b1-e5b4-4150-a014-b285a4f6f3f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.481932 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d419b1-e5b4-4150-a014-b285a4f6f3f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.483348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b914a9f5c7a431d889feaf0e32a4a14', 'request_id': 'req-ae2992f6-b13f-4185-842a-108fa4e7492b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:18.483504 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d419b1-e5b4-4150-a014-b285a4f6f3f6 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.492760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b914a9f5c7a431d889feaf0e32a4a14', 'request_id': 'req-ae2992f6-b13f-4185-842a-108fa4e7492b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:18.496365 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1415/2826] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:18.500466 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b914a9f5c7a431d889feaf0e32a4a14', 'request_id': 'req-ae2992f6-b13f-4185-842a-108fa4e7492b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:18.512546 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] The token cache is being invalidate because project 2b914a9f5c7a431d889feaf0e32a4a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:18.512839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:18.513517 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae2992f6-b13f-4185-842a-108fa4e7492b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:18.515130 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1412/2827] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:18 2022] DELETE /identity/v3/projects/2b914a9f5c7a431d889feaf0e32a4a14 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:18.521457 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544febe-d686-4e0c-80fd-3869245490c3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.521800 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544febe-d686-4e0c-80fd-3869245490c3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.522044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544febe-d686-4e0c-80fd-3869245490c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.543240 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7544febe-d686-4e0c-80fd-3869245490c3 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:18.544479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7544febe-d686-4e0c-80fd-3869245490c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:18.617855 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1416/2828] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:18.623883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2bab3-dd99-421c-bdbb-c7098aa36362 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.624344 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2bab3-dd99-421c-bdbb-c7098aa36362 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.624595 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2bab3-dd99-421c-bdbb-c7098aa36362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.647126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-efc2bab3-dd99-421c-bdbb-c7098aa36362 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:18.647509 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-efc2bab3-dd99-421c-bdbb-c7098aa36362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:18.655607 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1413/2829] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:18.660066 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544febe-d686-4e0c-80fd-3869245490c3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.674886 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544febe-d686-4e0c-80fd-3869245490c3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.680221 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.680941 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.681246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.681519 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.682165 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:18.682465 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.684063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.699667 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1417/2830] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:18.704520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2bab3-dd99-421c-bdbb-c7098aa36362 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.711251 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2bab3-dd99-421c-bdbb-c7098aa36362 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.717145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.717881 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.718252 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.718623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.719346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.721394 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.754158 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1414/2831] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:18 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:18.758219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.768878 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f627f-f2c0-4ab9-9fee-ddf95a78b299 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.808420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:18.834722 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.835368 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.835652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.835931 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.836640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.838653 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.862622 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1418/2832] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:18 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:18.866572 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.877313 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773ebc1-1080-4135-80a9-7fb8f3b05810 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.882514 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.883145 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.883420 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.883745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.884589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.886222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.900911 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1415/2833] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:18.902071 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.912565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668cac-3caa-4614-9889-c49093ac5674 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.917132 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.917721 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.917997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.918269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.918935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.920599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.932030 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1419/2834] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:12:18.936421 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.947088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a419c70-9109-4d70-a3c6-47b1211e496d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.952084 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcc196-1a49-4504-a7da-7c2b24b409cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.952792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcc196-1a49-4504-a7da-7c2b24b409cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.953074 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcc196-1a49-4504-a7da-7c2b24b409cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.953380 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcc196-1a49-4504-a7da-7c2b24b409cf admin admin] PATH_INFO: `/v3/projects/6deb6ef4f6d747649d0fa8d98f12ec77/users/e21a593c324a4b94820dbdb3ae5a3ed0/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.960257 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:18.962101 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:18.975691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcc196-1a49-4504-a7da-7c2b24b409cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6deb6ef4f6d747649d0fa8d98f12ec77, user_id=e21a593c324a4b94820dbdb3ae5a3ed0, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.976089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977b8b7-2e5a-470f-b032-044ad66fef9e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:18.980770 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcc196-1a49-4504-a7da-7c2b24b409cf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.980967 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:18.981420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:18.981700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:18.981978 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] PATH_INFO: `/v3/users/937f426b7053489c85cb50d92c3c358d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:18.983115 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=937f426b7053489c85cb50d92c3c358d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:18.985105 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:18.998070 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1416/2835] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:18 2022] PUT /identity/v3/projects/6deb6ef4f6d747649d0fa8d98f12ec77/users/e21a593c324a4b94820dbdb3ae5a3ed0/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:19.009958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:19.010234 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:19.010500 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:19.041381 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] MFA Rules not processed for user `e21a593c324a4b94820dbdb3ae5a3ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:19.042787 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:19.050616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937f426b7053489c85cb50d92c3c358d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.059283 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937f426b7053489c85cb50d92c3c358d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.073169 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937f426b7053489c85cb50d92c3c358d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.073907 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937f426b7053489c85cb50d92c3c358d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.074544 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-34287dd2-05a3-4365-83ac-23da3029af1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937f426b7053489c85cb50d92c3c358d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.088343 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1420/2836] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:18 2022] DELETE /identity/v3/users/937f426b7053489c85cb50d92c3c358d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:19.092952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:19.093235 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:19.093518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:19.096126 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:19.114458 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:19.115673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:19.118249 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1417/2837] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:19.129981 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:19.132104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:19.186262 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1421/2838] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:19.196311 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:19.197614 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:19.205958 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a52f7-6ab2-49b3-bac1-4e6d6410eca2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:19.229997 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:19.265485 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47392b40-f8e5-4d54-86a4-872b43450e9f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:19.267692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:19.268308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:19.268678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:19.268889 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:19.269209 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:19.269847 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:19.270254 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:19.270531 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:19.280836 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:19.282274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:19.293013 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1422/2839] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:12:19.340837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:19.342296 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:19.353794 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1418/2840] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:12:19.636589 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:19.639445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:19.662337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e361d46-eb36-41c1-ad4c-109421bd19a4 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:19.674051 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:19.675093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:19.675615 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:19.676813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] PATH_INFO: `/v3/projects/b1579c5a00e749999cf06f0551913b81` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:19.679021 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1579c5a00e749999cf06f0551913b81)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:19.683975 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:19.781913 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1579c5a00e749999cf06f0551913b81', 'request_id': 'req-2392e1a5-352f-4616-bca3-e0ca22f83698'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.790088 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1579c5a00e749999cf06f0551913b81', 'request_id': 'req-2392e1a5-352f-4616-bca3-e0ca22f83698'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.797265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1579c5a00e749999cf06f0551913b81', 'request_id': 'req-2392e1a5-352f-4616-bca3-e0ca22f83698'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.811226 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] The token cache is being invalidate because project b1579c5a00e749999cf06f0551913b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:19.811668 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.812498 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.814257 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1423/2841] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:19 2022] DELETE /identity/v3/projects/b1579c5a00e749999cf06f0551913b81 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:19.818531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:19.832506 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42760ac8-1b58-48d4-b639-479ccfa6035c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:19.929116 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:19.929760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:19.930043 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:19.930318 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] PATH_INFO: `/v3/users/a945e53b86614b079efdef3025080eae` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:19.931458 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a945e53b86614b079efdef3025080eae)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:19.933403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:19.934915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:19.983373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a945e53b86614b079efdef3025080eae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.984141 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a945e53b86614b079efdef3025080eae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:19.992753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a945e53b86614b079efdef3025080eae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.004899 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a945e53b86614b079efdef3025080eae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.005588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d976269-1a16-4c42-bff6-91ab3e29786c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a945e53b86614b079efdef3025080eae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.018459 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1419/2842] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:19 2022] DELETE /identity/v3/users/a945e53b86614b079efdef3025080eae => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:20.310883 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:20.316853 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:20.331443 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e1a5-352f-4616-bca3-e0ca22f83698 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:20.401880 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:20.402727 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.403127 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.403405 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] PATH_INFO: `/v3/projects/2a37180c429041e6992e30607ab811c3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.404700 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a37180c429041e6992e30607ab811c3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:20.406927 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:20.458015 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a37180c429041e6992e30607ab811c3', 'request_id': 'req-98003106-4318-4482-86e0-05babb3dd3e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.464354 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a37180c429041e6992e30607ab811c3', 'request_id': 'req-98003106-4318-4482-86e0-05babb3dd3e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.466373 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:20.471890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a37180c429041e6992e30607ab811c3', 'request_id': 'req-98003106-4318-4482-86e0-05babb3dd3e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.485843 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] The token cache is being invalidate because project 2a37180c429041e6992e30607ab811c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:20.486135 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.486762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-98003106-4318-4482-86e0-05babb3dd3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:20.488340 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1424/2843] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:20 2022] DELETE /identity/v3/projects/2a37180c429041e6992e30607ab811c3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:20.493602 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced5d4b-b815-4acb-8a8f-99c97e72519c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.493883 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced5d4b-b815-4acb-8a8f-99c97e72519c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.494158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced5d4b-b815-4acb-8a8f-99c97e72519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.516326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3ced5d4b-b815-4acb-8a8f-99c97e72519c None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:20.517589 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ced5d4b-b815-4acb-8a8f-99c97e72519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:20.596078 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1420/2844] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:20.606412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68078996-5787-4fd1-9394-baeeb8f0f0b9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.606794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68078996-5787-4fd1-9394-baeeb8f0f0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.607077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68078996-5787-4fd1-9394-baeeb8f0f0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.630371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-68078996-5787-4fd1-9394-baeeb8f0f0b9 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:20.631670 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68078996-5787-4fd1-9394-baeeb8f0f0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:20.639820 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1425/2845] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:20.644417 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced5d4b-b815-4acb-8a8f-99c97e72519c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:20.654948 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced5d4b-b815-4acb-8a8f-99c97e72519c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:20.659969 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:20.661388 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.661666 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.661967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.662674 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:20.662979 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:20.664710 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:20.678067 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1421/2846] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:20.682656 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68078996-5787-4fd1-9394-baeeb8f0f0b9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:20.693110 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68078996-5787-4fd1-9394-baeeb8f0f0b9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:20.697841 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80a9ad-326f-48e6-9ebf-370f3d1d8023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:20.698573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80a9ad-326f-48e6-9ebf-370f3d1d8023 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.698743 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80a9ad-326f-48e6-9ebf-370f3d1d8023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.699000 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80a9ad-326f-48e6-9ebf-370f3d1d8023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.699637 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80a9ad-326f-48e6-9ebf-370f3d1d8023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:20.701613 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80a9ad-326f-48e6-9ebf-370f3d1d8023 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:20.734981 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1426/2847] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:20 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:20.736721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:20.747592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b739b-a3e1-4a77-ad90-5eb74506b81c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:20.790543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f38692-7000-40a1-a8ed-7eff26e75b59 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.791863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f38692-7000-40a1-a8ed-7eff26e75b59 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.791863 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f38692-7000-40a1-a8ed-7eff26e75b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.811842 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a2f38692-7000-40a1-a8ed-7eff26e75b59 None None] MFA Rules not processed for user `e21a593c324a4b94820dbdb3ae5a3ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:20.813199 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2f38692-7000-40a1-a8ed-7eff26e75b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:20.814142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:20.814762 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.815222 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.815505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.816158 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:20.818152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:20.833920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:20.857056 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1422/2848] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:20 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:20.858083 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:20.869825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb298-4b96-4168-b844-5ff02bfcc931 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:20.875246 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:20.875774 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.876198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.876640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.877323 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:20.879017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:20.883732 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1427/2849] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:20.892214 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1423/2850] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:12:20.895979 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f38692-7000-40a1-a8ed-7eff26e75b59 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:20.907973 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f38692-7000-40a1-a8ed-7eff26e75b59 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:20.913459 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261b401-f59d-4114-8223-1d40521fa6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:20.914063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261b401-f59d-4114-8223-1d40521fa6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.914339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261b401-f59d-4114-8223-1d40521fa6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.914612 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261b401-f59d-4114-8223-1d40521fa6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.915334 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261b401-f59d-4114-8223-1d40521fa6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:20.917248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261b401-f59d-4114-8223-1d40521fa6e3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:20.932425 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1428/2851] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:12:20.934527 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:20.945709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bb0c1-b700-487f-959d-c808342dba45 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:20.950661 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:20.951353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.951635 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.951902 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] PATH_INFO: `/v3/projects/a70db418921c443284eb02241faab2b1/users/5d5a3967fe204589ab0ffd3771b56d0c/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:20.968577 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70db418921c443284eb02241faab2b1, user_id=5d5a3967fe204589ab0ffd3771b56d0c, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:20.971201 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:20.987098 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1424/2852] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:20 2022] PUT /identity/v3/projects/a70db418921c443284eb02241faab2b1/users/5d5a3967fe204589ab0ffd3771b56d0c/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:20.997558 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13c63f-f9a7-4c19-b543-dc557a1d538b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:20.997848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13c63f-f9a7-4c19-b543-dc557a1d538b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:20.998126 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13c63f-f9a7-4c19-b543-dc557a1d538b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.027575 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4e13c63f-f9a7-4c19-b543-dc557a1d538b None None] MFA Rules not processed for user `5d5a3967fe204589ab0ffd3771b56d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:21.028942 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4e13c63f-f9a7-4c19-b543-dc557a1d538b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:21.098277 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1429/2853] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:21.102970 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.174537 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e509d50-91c6-41be-92bc-d8561f4df069 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.181939 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.182676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.182956 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.183232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.183893 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.185850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.215806 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1425/2854] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:21 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:21.220324 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13c63f-f9a7-4c19-b543-dc557a1d538b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.232012 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13c63f-f9a7-4c19-b543-dc557a1d538b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.280751 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:21.303627 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.304372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.304881 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.305282 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.306047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.308312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.332757 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1430/2855] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:21 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:21.337037 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.349704 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0975db-2350-41e5-ab41-5ae8e11b6865 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.355015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.355713 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.356070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.356438 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.357157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.358845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.376006 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.380716 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1426/2856] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:21.389077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbe762-9ef4-4bcc-8918-7c05b34cf125 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.396156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.397067 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.397356 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.397635 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] PATH_INFO: `/v3/projects/e9be9b53c0a44a24816802b268436566/users/98219c840d3d492c9440fc7f26c1ce88/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.408670 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.410734 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:21.425635 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9be9b53c0a44a24816802b268436566, user_id=98219c840d3d492c9440fc7f26c1ce88, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.428538 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.444288 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1431/2857] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:21 2022] PUT /identity/v3/projects/e9be9b53c0a44a24816802b268436566/users/98219c840d3d492c9440fc7f26c1ce88/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:21.448488 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.490933 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930fcd3-41b3-41f9-998f-693d236dbf8b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.521020 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d24bb-0e1b-4df5-a1cc-03ead0fcb052 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.527651 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.528323 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.530420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.530420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.530420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.531410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.543019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:21.544307 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1432/2858] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:21.549830 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.561672 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383ae8b-8a1f-4e28-ba20-a42791395a45 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.566682 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb30a-f93a-460d-a8a6-07c7ad98c700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.567416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb30a-f93a-460d-a8a6-07c7ad98c700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.567741 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb30a-f93a-460d-a8a6-07c7ad98c700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.567998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb30a-f93a-460d-a8a6-07c7ad98c700 admin admin] PATH_INFO: `/v3/projects/e9be9b53c0a44a24816802b268436566/users/98219c840d3d492c9440fc7f26c1ce88/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.570401 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb30a-f93a-460d-a8a6-07c7ad98c700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9be9b53c0a44a24816802b268436566, user_id=98219c840d3d492c9440fc7f26c1ce88, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.573336 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb30a-f93a-460d-a8a6-07c7ad98c700 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.583241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.583241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.583241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.583241 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.585817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:21.586246 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1433/2859] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:21 2022] PUT /identity/v3/projects/e9be9b53c0a44a24816802b268436566/users/98219c840d3d492c9440fc7f26c1ce88/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:21.599265 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.599552 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.599837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.607540 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.610538 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.625608 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1427/2860] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:12:21.634004 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] MFA Rules not processed for user `98219c840d3d492c9440fc7f26c1ce88`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:21.635208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:21.709630 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1434/2861] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:21.720429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:21.721988 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:21.788239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2455b4-ab69-4d3d-8d16-40d744aec38e service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:21.842823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:21.843428 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:21.843721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:21.844132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:21.917406 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:21.919642 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:21.930259 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1428/2862] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:24.388784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:24.390227 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:24.417089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:24.490812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7653ed-0bdc-42cc-a5e6-1da7e823a371 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:24.496935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2de78a-ae4d-4b13-b09a-1620eabc429f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:24.497683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2de78a-ae4d-4b13-b09a-1620eabc429f service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:24.497988 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2de78a-ae4d-4b13-b09a-1620eabc429f service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:24.498270 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2de78a-ae4d-4b13-b09a-1620eabc429f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:24.500154 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3f2de78a-ae4d-4b13-b09a-1620eabc429f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:24.514355 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2de78a-ae4d-4b13-b09a-1620eabc429f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:24.516089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2de78a-ae4d-4b13-b09a-1620eabc429f service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:24.530897 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1435/2863] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:24.554205 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:24.603576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:24.628606 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc3f37-1d5c-43f5-98f8-da8494a73f96 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:24.633403 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:24.634151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:24.634473 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:24.634792 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] PATH_INFO: `/v3/users/5d5a3967fe204589ab0ffd3771b56d0c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:24.635823 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d5a3967fe204589ab0ffd3771b56d0c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:24.637765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:24.689712 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d5a3967fe204589ab0ffd3771b56d0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:24.690451 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d5a3967fe204589ab0ffd3771b56d0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:24.698651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d5a3967fe204589ab0ffd3771b56d0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:24.712591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d5a3967fe204589ab0ffd3771b56d0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:24.713343 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d5a3967fe204589ab0ffd3771b56d0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:24.729517 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1429/2864] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:24 2022] DELETE /identity/v3/users/5d5a3967fe204589ab0ffd3771b56d0c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:24.734134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:24.734530 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:24.734888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:24.755884 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:24.757153 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:24.827087 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1436/2865] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:24.839459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:24.842019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:24.920055 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163b44-2ffa-4fb3-b74b-e1b733bea118 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:24.925069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:24.927142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:24.927676 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:24.927895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:24.939193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:24.940671 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:24.952060 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1430/2866] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:25.248705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:25.250185 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:25.262664 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114858ad-cc80-47cc-87d9-99190e502dfe None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:25.267314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:25.267983 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:25.268261 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:25.268565 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] PATH_INFO: `/v3/projects/a70db418921c443284eb02241faab2b1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:25.270263 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70db418921c443284eb02241faab2b1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:25.272217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:25.330176 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70db418921c443284eb02241faab2b1', 'request_id': 'req-11620a50-37dd-469a-91ed-50edb3c17fe1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.336546 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70db418921c443284eb02241faab2b1', 'request_id': 'req-11620a50-37dd-469a-91ed-50edb3c17fe1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.342489 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70db418921c443284eb02241faab2b1', 'request_id': 'req-11620a50-37dd-469a-91ed-50edb3c17fe1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.355237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] The token cache is being invalidate because project a70db418921c443284eb02241faab2b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:25.355525 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.356156 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.357676 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1437/2867] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:25 2022] DELETE /identity/v3/projects/a70db418921c443284eb02241faab2b1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:25.361699 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:25.372499 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce347fb-3384-4326-9872-5a0effbaab8f service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:25.435255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:25.435888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:25.436220 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:25.436467 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] PATH_INFO: `/v3/users/98219c840d3d492c9440fc7f26c1ce88` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:25.437863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98219c840d3d492c9440fc7f26c1ce88)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:25.439894 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:25.442327 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:25.491104 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98219c840d3d492c9440fc7f26c1ce88'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.491753 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98219c840d3d492c9440fc7f26c1ce88'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.499135 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98219c840d3d492c9440fc7f26c1ce88'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.511785 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98219c840d3d492c9440fc7f26c1ce88'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.512531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ed6c7a55-a9cc-4b09-8d36-75edfe148e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98219c840d3d492c9440fc7f26c1ce88'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.525969 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1431/2868] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:25 2022] DELETE /identity/v3/users/98219c840d3d492c9440fc7f26c1ce88 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:25.810823 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:25.812473 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:25.830688 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11620a50-37dd-469a-91ed-50edb3c17fe1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:25.852463 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:25.894236 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:25.895026 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:25.895316 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:25.895606 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] PATH_INFO: `/v3/projects/e9be9b53c0a44a24816802b268436566` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:25.896759 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9be9b53c0a44a24816802b268436566)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:25.898756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:25.946625 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9be9b53c0a44a24816802b268436566', 'request_id': 'req-937cb745-0506-4a2e-b628-7a61e55504c4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.952779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9be9b53c0a44a24816802b268436566', 'request_id': 'req-937cb745-0506-4a2e-b628-7a61e55504c4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.958496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9be9b53c0a44a24816802b268436566', 'request_id': 'req-937cb745-0506-4a2e-b628-7a61e55504c4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.971053 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] The token cache is being invalidate because project e9be9b53c0a44a24816802b268436566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:25.971341 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.971977 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-937cb745-0506-4a2e-b628-7a61e55504c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:25.974027 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1438/2869] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:25 2022] DELETE /identity/v3/projects/e9be9b53c0a44a24816802b268436566 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:25.979173 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c110c7-8dd2-4c12-ac80-eeb5bca7cafd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:25.979633 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c110c7-8dd2-4c12-ac80-eeb5bca7cafd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:25.979731 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c110c7-8dd2-4c12-ac80-eeb5bca7cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.000367 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-81c110c7-8dd2-4c12-ac80-eeb5bca7cafd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:26.001520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81c110c7-8dd2-4c12-ac80-eeb5bca7cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:26.065684 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1432/2870] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:26.073641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedb36a-4ae7-458e-a5be-c9331f010edd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.073955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedb36a-4ae7-458e-a5be-c9331f010edd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.074260 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedb36a-4ae7-458e-a5be-c9331f010edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.095385 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6dedb36a-4ae7-458e-a5be-c9331f010edd None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:26.096590 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6dedb36a-4ae7-458e-a5be-c9331f010edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:26.106275 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1439/2871] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:26.110688 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c110c7-8dd2-4c12-ac80-eeb5bca7cafd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.120495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c110c7-8dd2-4c12-ac80-eeb5bca7cafd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.125307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.126307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.126307 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.126600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.127070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:26.128085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.129709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.142382 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1433/2872] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:26.147362 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedb36a-4ae7-458e-a5be-c9331f010edd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.157416 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedb36a-4ae7-458e-a5be-c9331f010edd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.163295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.163871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.164157 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.164497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.165177 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.167147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.199163 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1440/2873] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:26 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:26.200291 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.210904 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde89a5-6d99-4804-b65c-90089bb4865b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.267980 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.268684 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.269009 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.269886 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.270569 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.272608 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.283874 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:26.298628 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1434/2874] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:26 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:26.302640 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.313641 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43100283-7bd0-4338-a59c-29794e4d551e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.318414 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.319027 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.319310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.319585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.320237 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.321902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.333169 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1441/2875] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:26.337261 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.353183 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcc714-45d4-4f2c-b40e-2210f9f2c5b6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.357907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e8a01-05fc-4f68-9a9d-d8a4aab5e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.358532 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e8a01-05fc-4f68-9a9d-d8a4aab5e440 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.358812 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e8a01-05fc-4f68-9a9d-d8a4aab5e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.359086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e8a01-05fc-4f68-9a9d-d8a4aab5e440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.359732 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e8a01-05fc-4f68-9a9d-d8a4aab5e440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.361345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e8a01-05fc-4f68-9a9d-d8a4aab5e440 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.372815 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1435/2876] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:26.377091 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.388164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb929f-9fa4-4cbb-bfba-917a112e77d2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.392909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.393569 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.393854 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.394655 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] PATH_INFO: `/v3/projects/6dbef97f3e574703ad855bec6b95cb70/users/c7d27afe21614a769c73b1273a0b5e15/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.413372 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dbef97f3e574703ad855bec6b95cb70, user_id=c7d27afe21614a769c73b1273a0b5e15, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.416174 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.431871 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1442/2877] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:26 2022] PUT /identity/v3/projects/6dbef97f3e574703ad855bec6b95cb70/users/c7d27afe21614a769c73b1273a0b5e15/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:26.442809 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59e5e-54a5-4f02-a1a2-37f4a9fc3114 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.443092 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59e5e-54a5-4f02-a1a2-37f4a9fc3114 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.443362 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f59e5e-54a5-4f02-a1a2-37f4a9fc3114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.473003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-35f59e5e-54a5-4f02-a1a2-37f4a9fc3114 None None] MFA Rules not processed for user `c7d27afe21614a769c73b1273a0b5e15`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:26.474212 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-35f59e5e-54a5-4f02-a1a2-37f4a9fc3114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:26.539873 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1436/2878] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:26.544365 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.596091 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:26.613093 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5ddc-514c-4ae0-b64f-546ce8ff2514 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.617943 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.618555 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.618840 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.619143 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.619776 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.621734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.651633 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1443/2879] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:26 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:26.655758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59e5e-54a5-4f02-a1a2-37f4a9fc3114 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.666799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f59e5e-54a5-4f02-a1a2-37f4a9fc3114 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.728157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.728819 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.729099 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.729374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.730028 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.731908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.755516 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1437/2880] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:26 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:26.760043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.771108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a653c7-b239-4af4-9193-19facc4f2a9c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.776524 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.777159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.777477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.777754 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.778429 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.780096 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.792602 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1444/2881] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:26.796709 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.809036 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b369ae-42f2-41bf-9884-9c5801c4e07f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.814748 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b694c6f-1473-4568-ae6e-82553e7a446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.815531 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b694c6f-1473-4568-ae6e-82553e7a446f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.815866 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b694c6f-1473-4568-ae6e-82553e7a446f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.816832 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b694c6f-1473-4568-ae6e-82553e7a446f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.817498 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b694c6f-1473-4568-ae6e-82553e7a446f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.819346 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b694c6f-1473-4568-ae6e-82553e7a446f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.831120 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1438/2882] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:26.836464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:26.848003 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5d14f-0be5-4e1b-bc12-025898f0bb71 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:26.852748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:26.853452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.853733 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.854010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] PATH_INFO: `/v3/projects/b1bfcbb4377447229d781c79bc70d453/users/b9da517178264ac5b7b0926593aa7af6/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.870445 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bfcbb4377447229d781c79bc70d453, user_id=b9da517178264ac5b7b0926593aa7af6, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:26.873218 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:26.887835 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1445/2883] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:26 2022] PUT /identity/v3/projects/b1bfcbb4377447229d781c79bc70d453/users/b9da517178264ac5b7b0926593aa7af6/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:26.899560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44948f8c-fcc8-47f6-8111-4cf29c3aaee6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:26.900888 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44948f8c-fcc8-47f6-8111-4cf29c3aaee6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:26.901221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44948f8c-fcc8-47f6-8111-4cf29c3aaee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:26.934591 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-44948f8c-fcc8-47f6-8111-4cf29c3aaee6 None None] MFA Rules not processed for user `b9da517178264ac5b7b0926593aa7af6`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:26.935813 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-44948f8c-fcc8-47f6-8111-4cf29c3aaee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:26.982913 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:27.009518 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1439/2884] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:27.014192 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:27.079375 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f5903-59eb-4236-9173-c4690c3afa5f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:27.085337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:27.085998 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.086289 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.086563 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.087215 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:27.089410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:27.126621 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1446/2885] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:27 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:27.131070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44948f8c-fcc8-47f6-8111-4cf29c3aaee6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:27.141967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44948f8c-fcc8-47f6-8111-4cf29c3aaee6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:27.199908 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:27.200558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.200839 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.201112 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.201739 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:27.203716 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:27.226730 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1440/2886] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:27 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:27.230734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:27.241417 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddaa4c2-2879-433c-b4e4-d5bf15d3592b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:27.246159 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:27.246762 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.247044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.247369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.247972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:27.249631 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:27.261353 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1447/2887] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:27.265617 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:27.277398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e6b9-9972-464f-9837-5c167049b546 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:27.282137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:27.282802 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.283080 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.283355 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] PATH_INFO: `/v3/projects/1fb5c8bdbab44105870cfb021d94a98a/users/068119ff203847ceb0c43dfbd0f9f1f9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.300947 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb5c8bdbab44105870cfb021d94a98a, user_id=068119ff203847ceb0c43dfbd0f9f1f9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:27.303650 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:27.320914 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1441/2888] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:27 2022] PUT /identity/v3/projects/1fb5c8bdbab44105870cfb021d94a98a/users/068119ff203847ceb0c43dfbd0f9f1f9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:27.325242 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:27.336456 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81e16-7304-4daf-8762-7ce935c3098f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:27.402420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcb6fd-734d-4d4d-ac05-76480d2d3aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:27.402420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcb6fd-734d-4d4d-ac05-76480d2d3aed admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.402420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcb6fd-734d-4d4d-ac05-76480d2d3aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.402420 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcb6fd-734d-4d4d-ac05-76480d2d3aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.403017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcb6fd-734d-4d4d-ac05-76480d2d3aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:27.403017 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcb6fd-734d-4d4d-ac05-76480d2d3aed admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:27.416657 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1448/2889] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:27.420855 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:27.431896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951d819-d8a5-4ee3-8585-55b21645008b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:27.437967 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:27.438636 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.438920 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.439196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] PATH_INFO: `/v3/projects/1fb5c8bdbab44105870cfb021d94a98a/users/068119ff203847ceb0c43dfbd0f9f1f9/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.441791 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb5c8bdbab44105870cfb021d94a98a, user_id=068119ff203847ceb0c43dfbd0f9f1f9, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:27.444656 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:27.456470 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1442/2890] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:27 2022] PUT /identity/v3/projects/1fb5c8bdbab44105870cfb021d94a98a/users/068119ff203847ceb0c43dfbd0f9f1f9/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:27.467652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.467985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.468205 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.489007 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:27.500789 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] MFA Rules not processed for user `068119ff203847ceb0c43dfbd0f9f1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:27.502055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:27.572900 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1449/2891] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:27.584358 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:27.586117 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:27.649170 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc52f7-ecfd-40f1-a5e2-e8a1fcf01ba6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:27.652859 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:27.653462 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:27.653737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:27.654010 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:27.711811 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:27.723001 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:27.724487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:27.733950 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1443/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:29.256958 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:29.258408 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:29.271148 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a3eaa-eb5e-4ef6-a3b9-ec866cf3fc10 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:29.274802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:29.275406 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:29.275692 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:29.275962 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:29.337281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:29.338748 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:29.348637 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1450/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:29.636457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:29.638022 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:29.754435 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359abb7-00b0-403c-9110-31b5e327ed14 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:29.761615 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:29.762357 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:29.762773 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:29.763151 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:29.773588 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:29.787142 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:29.788994 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:29.801527 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1444/2894] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:29.959108 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:29.969419 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82811842-80ec-4cd0-b80a-26eac0df0194 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:29.974314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e711ae-231a-45e2-9175-95519123bc8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '009181b0acc24632a5614482cdebb970', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:29.974944 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e711ae-231a-45e2-9175-95519123bc8b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:29.975312 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e711ae-231a-45e2-9175-95519123bc8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:29.975570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e711ae-231a-45e2-9175-95519123bc8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:29.985909 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e711ae-231a-45e2-9175-95519123bc8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:29.987369 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e711ae-231a-45e2-9175-95519123bc8b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:29.998276 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1451/2895] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:33.938825 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:34.004810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f8d34-3fe8-4e0a-9e8a-317c3eebb17a service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:34.009515 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:34.010302 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:34.010667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:34.010897 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] PATH_INFO: `/v3/users/c7d27afe21614a769c73b1273a0b5e15` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:34.012045 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d27afe21614a769c73b1273a0b5e15)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:34.014032 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:34.067263 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d27afe21614a769c73b1273a0b5e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.068008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d27afe21614a769c73b1273a0b5e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.075850 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d27afe21614a769c73b1273a0b5e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.088751 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d27afe21614a769c73b1273a0b5e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.089495 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d27afe21614a769c73b1273a0b5e15'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.092750 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:34.106406 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1445/2896] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:33 2022] DELETE /identity/v3/users/c7d27afe21614a769c73b1273a0b5e15 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:34.110851 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:34.111638 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:34.111915 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:34.132790 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:34.134050 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:34.196319 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1452/2897] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:34.208488 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:34.210042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:34.273230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffab56c-8ba7-429c-bfb8-0b9da4dd044c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:34.277274 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:34.278124 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:34.278399 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:34.278678 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:34.289213 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:34.290649 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:34.301361 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1446/2898] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:34.810138 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:34.818047 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:34.831208 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59fe3e-8a42-446e-b401-5e3ab317a1a7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:34.836757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:34.837447 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:34.837784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:34.838005 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] PATH_INFO: `/v3/projects/6dbef97f3e574703ad855bec6b95cb70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:34.839190 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dbef97f3e574703ad855bec6b95cb70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:34.841482 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:34.858756 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:34.892737 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dbef97f3e574703ad855bec6b95cb70', 'request_id': 'req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.899812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dbef97f3e574703ad855bec6b95cb70', 'request_id': 'req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.905645 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dbef97f3e574703ad855bec6b95cb70', 'request_id': 'req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.919367 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] The token cache is being invalidate because project 6dbef97f3e574703ad855bec6b95cb70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:34.919885 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.920599 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:34.922118 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1453/2899] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:34 2022] DELETE /identity/v3/projects/6dbef97f3e574703ad855bec6b95cb70 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:34.926293 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:34.936619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c568bd9-60dc-4094-a569-c2457251c94f service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:35.001069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:35.001771 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:35.001978 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:35.002248 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] PATH_INFO: `/v3/users/b9da517178264ac5b7b0926593aa7af6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:35.003896 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9da517178264ac5b7b0926593aa7af6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:35.005915 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:35.056879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9da517178264ac5b7b0926593aa7af6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.057623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9da517178264ac5b7b0926593aa7af6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.066910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9da517178264ac5b7b0926593aa7af6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.080453 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9da517178264ac5b7b0926593aa7af6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.082150 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9da517178264ac5b7b0926593aa7af6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.098953 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1447/2900] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:34 2022] DELETE /identity/v3/users/b9da517178264ac5b7b0926593aa7af6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:35.391802 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:35.393382 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:35.419784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe512bbc-cae5-46d6-bb59-93d8b88a17d6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:35.480044 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:35.480648 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:35.480947 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:35.481219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] PATH_INFO: `/v3/projects/b1bfcbb4377447229d781c79bc70d453` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:35.482339 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bfcbb4377447229d781c79bc70d453)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:35.484230 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:35.535327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bfcbb4377447229d781c79bc70d453', 'request_id': 'req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.541968 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bfcbb4377447229d781c79bc70d453', 'request_id': 'req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.547703 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bfcbb4377447229d781c79bc70d453', 'request_id': 'req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.560765 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] The token cache is being invalidate because project b1bfcbb4377447229d781c79bc70d453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:35.561113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.561784 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cbe082d9-9eb5-4d57-9d52-b984f0c50f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.563377 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1454/2901] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:35 2022] DELETE /identity/v3/projects/b1bfcbb4377447229d781c79bc70d453 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:35.567567 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:35.569058 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:35.582019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adf825-b5d9-46df-a5c9-b27ab8e4f47d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:35.593793 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:35.643730 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:35.644353 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:35.644667 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:35.644945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] PATH_INFO: `/v3/users/068119ff203847ceb0c43dfbd0f9f1f9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:35.646076 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=068119ff203847ceb0c43dfbd0f9f1f9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:35.647993 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:35.696669 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068119ff203847ceb0c43dfbd0f9f1f9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.697398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068119ff203847ceb0c43dfbd0f9f1f9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.704697 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068119ff203847ceb0c43dfbd0f9f1f9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.717297 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068119ff203847ceb0c43dfbd0f9f1f9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.718019 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068119ff203847ceb0c43dfbd0f9f1f9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:35.731330 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1448/2902] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:35 2022] DELETE /identity/v3/users/068119ff203847ceb0c43dfbd0f9f1f9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:36.021673 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:36.021959 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:36.022232 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:36.043442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:36.044968 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:36.046125 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:36.047307 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:36.087809 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:36.118221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3056f7-136e-4dd4-9290-c85c56310f4a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:36.120184 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1455/2903] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:12:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:36.124579 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:36.125299 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:36.125651 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:36.125927 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] PATH_INFO: `/v3/projects/1fb5c8bdbab44105870cfb021d94a98a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:36.127088 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb5c8bdbab44105870cfb021d94a98a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:36.129176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:36.169994 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:36.171425 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:36.172093 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:36.182833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb5c8bdbab44105870cfb021d94a98a', 'request_id': 'req-6bf8c366-f150-4843-ba66-f600510065ca'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:36.191691 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb5c8bdbab44105870cfb021d94a98a', 'request_id': 'req-6bf8c366-f150-4843-ba66-f600510065ca'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:36.198326 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb5c8bdbab44105870cfb021d94a98a', 'request_id': 'req-6bf8c366-f150-4843-ba66-f600510065ca'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:36.210520 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] The token cache is being invalidate because project 1fb5c8bdbab44105870cfb021d94a98a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:36.210810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:36.211471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6bf8c366-f150-4843-ba66-f600510065ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:36.212974 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1449/2904] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:36 2022] DELETE /identity/v3/projects/1fb5c8bdbab44105870cfb021d94a98a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:36.248982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31749810-28ad-4daf-b18b-81b50b43a5f2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:36.327920 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:36.328585 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:36.328867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:36.329164 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:36.331089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:36.404352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:36.404352 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:36.416333 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1456/2905] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:12:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 25 01:12:36.955127 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d800e9-f763-4b1c-98ad-2d8694a2c166 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:36.955414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d800e9-f763-4b1c-98ad-2d8694a2c166 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:36.955693 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d800e9-f763-4b1c-98ad-2d8694a2c166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:36.978600 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-81d800e9-f763-4b1c-98ad-2d8694a2c166 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:36.980003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81d800e9-f763-4b1c-98ad-2d8694a2c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:36.987416 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1450/2906] 10.222.0.26 () {58 vars in 967 bytes} [Sun Sep 25 01:12:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:37.000660 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:37.002078 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:37.015818 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38568806-0f9f-4d2b-8d54-becda1476277 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:37.021518 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:37.022274 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:37.022665 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:37.023039 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:37.032147 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:37.033783 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:37.043383 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1457/2907] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:12:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 25 01:12:37.295508 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074b70-7120-4514-91a8-b7b28db7df28 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:37.296069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074b70-7120-4514-91a8-b7b28db7df28 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:37.296069 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37074b70-7120-4514-91a8-b7b28db7df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:37.318187 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-37074b70-7120-4514-91a8-b7b28db7df28 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:37.319560 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-37074b70-7120-4514-91a8-b7b28db7df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:37.328221 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1451/2908] 10.222.0.26 () {58 vars in 1009 bytes} [Sun Sep 25 01:12:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:37.340554 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:37.353063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e19bc9-6c8e-40be-b370-59aa11c29485 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:37.358113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:37.358831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:37.359118 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:37.359392 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:37.365705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:37.368325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:37.370786 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:37.379339 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1458/2909] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:12:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:12:37.740698 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:37.740984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:37.741255 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:37.761640 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] MFA Rules not processed for user `616d85945f3d41839443b011bb48c412`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:37.762833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:37.769744 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1452/2910] 10.222.0.26 () {58 vars in 967 bytes} [Sun Sep 25 01:12:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 25 01:12:37.782410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:37.794719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810393a-5899-4f98-aa8a-35b83df213f5 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:37.800643 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9170b02-a90a-437e-a2cc-a609bfae9ae3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58594193ba6c41329a0bd1ca8205c69a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:37.803189 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9170b02-a90a-437e-a2cc-a609bfae9ae3 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:37.803477 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9170b02-a90a-437e-a2cc-a609bfae9ae3 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:37.803824 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9170b02-a90a-437e-a2cc-a609bfae9ae3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:37.814571 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9170b02-a90a-437e-a2cc-a609bfae9ae3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:37.816292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9170b02-a90a-437e-a2cc-a609bfae9ae3 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:37.824922 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1459/2911] 10.222.0.26 () {60 vars in 1384 bytes} [Sun Sep 25 01:12:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 25 01:12:40.215867 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:40.217845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:40.283414 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fec8e6-cef5-4c8c-a193-1a3e76e80562 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:40.288558 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:40.289193 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:40.289471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:40.289745 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] PATH_INFO: `/v3/users/e21a593c324a4b94820dbdb3ae5a3ed0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:40.290879 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21a593c324a4b94820dbdb3ae5a3ed0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:40.292829 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:40.343308 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21a593c324a4b94820dbdb3ae5a3ed0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.343960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21a593c324a4b94820dbdb3ae5a3ed0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.351303 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21a593c324a4b94820dbdb3ae5a3ed0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.364119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21a593c324a4b94820dbdb3ae5a3ed0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.364856 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21a593c324a4b94820dbdb3ae5a3ed0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.377873 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1453/2912] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:12:40 2022] DELETE /identity/v3/users/e21a593c324a4b94820dbdb3ae5a3ed0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:40.382955 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:40.383415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:40.383536 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:40.402658 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:40.403871 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:40.465653 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1460/2913] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:40.475701 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:40.477152 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:40.506623 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:40.542189 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d426d-ad55-4349-89fa-2bcb27e45574 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:40.545810 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be773492-9ffa-4081-a9c0-b42c05e6ca27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:40.546408 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be773492-9ffa-4081-a9c0-b42c05e6ca27 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:40.546687 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be773492-9ffa-4081-a9c0-b42c05e6ca27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:40.546976 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be773492-9ffa-4081-a9c0-b42c05e6ca27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:40.556760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be773492-9ffa-4081-a9c0-b42c05e6ca27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:40.558292 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be773492-9ffa-4081-a9c0-b42c05e6ca27 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:40.568484 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1454/2914] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:40.748764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:40.750217 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:40.764778 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f6fc2-75f9-4d1d-9870-1740c9063adb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:40.769904 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:40.770588 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:40.770870 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:40.771145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] PATH_INFO: `/v3/projects/6deb6ef4f6d747649d0fa8d98f12ec77` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:40.772315 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6deb6ef4f6d747649d0fa8d98f12ec77)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:40.775494 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:40.826794 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6deb6ef4f6d747649d0fa8d98f12ec77', 'request_id': 'req-2cecb510-49e9-4d5b-915f-30f6f18078dc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.833654 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6deb6ef4f6d747649d0fa8d98f12ec77', 'request_id': 'req-2cecb510-49e9-4d5b-915f-30f6f18078dc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.839415 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6deb6ef4f6d747649d0fa8d98f12ec77', 'request_id': 'req-2cecb510-49e9-4d5b-915f-30f6f18078dc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.852834 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] The token cache is being invalidate because project 6deb6ef4f6d747649d0fa8d98f12ec77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:12:40.853085 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.853768 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cecb510-49e9-4d5b-915f-30f6f18078dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:12:40.855344 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1461/2915] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:12:40 2022] DELETE /identity/v3/projects/6deb6ef4f6d747649d0fa8d98f12ec77 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 25 01:12:40.860571 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dce260-9d42-4920-82cf-f415ed070bd7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:40.860861 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dce260-9d42-4920-82cf-f415ed070bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:40.861133 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dce260-9d42-4920-82cf-f415ed070bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:40.883310 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b7dce260-9d42-4920-82cf-f415ed070bd7 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:40.884725 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b7dce260-9d42-4920-82cf-f415ed070bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:40.945821 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1455/2916] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:41.010875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5e10a-95cc-49ab-98dc-a5f1430d2f03 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.011079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5e10a-95cc-49ab-98dc-a5f1430d2f03 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.011330 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5e10a-95cc-49ab-98dc-a5f1430d2f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.036928 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1db5e10a-95cc-49ab-98dc-a5f1430d2f03 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:41.038142 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1db5e10a-95cc-49ab-98dc-a5f1430d2f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:41.046003 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1462/2917] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:41.051053 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dce260-9d42-4920-82cf-f415ed070bd7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:41.060960 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dce260-9d42-4920-82cf-f415ed070bd7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:41.065503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:41.066115 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.066386 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.066652 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.067287 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 25 01:12:41.067601 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:41.069195 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:41.081609 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1456/2918] 10.222.0.26 () {58 vars in 1153 bytes} [Sun Sep 25 01:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 25 01:12:41.086543 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5e10a-95cc-49ab-98dc-a5f1430d2f03 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:41.095806 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5e10a-95cc-49ab-98dc-a5f1430d2f03 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:41.101371 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:41.101972 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.102284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.102560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.103206 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:41.105170 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:41.136172 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1463/2919] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:41 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:41.140348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:41.151198 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f232a6-88c1-4fd0-98f3-f410fab5b8ae admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:41.209459 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:41.211060 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.211208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.211487 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.212132 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:41.214071 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:41.240300 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1457/2920] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:41 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 25 01:12:41.241803 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:41.251890 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc483943-ce1f-4a52-a55e-46f23e3a4fa9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:41.257409 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:41.258018 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.258292 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.258556 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.259193 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:41.260812 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:41.271728 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1464/2921] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:41.275934 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:41.286910 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18787f-bf26-4efc-9214-d9bb7de39943 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:41.291702 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d1cf2-12c6-43c2-9458-6d624215c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:41.292262 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d1cf2-12c6-43c2-9458-6d624215c8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.292522 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d1cf2-12c6-43c2-9458-6d624215c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.292796 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d1cf2-12c6-43c2-9458-6d624215c8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.293437 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d1cf2-12c6-43c2-9458-6d624215c8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:41.295017 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d1cf2-12c6-43c2-9458-6d624215c8a0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:41.306111 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1458/2922] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:41.310533 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:41.320813 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a3763-9e56-4a7c-a65d-bddff2f2fd60 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:41.325770 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:41.326418 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.326693 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.326966 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] PATH_INFO: `/v3/projects/93735199d2524322a000a8f09ee05f7b/users/303c2c84cd274f25bd61faf0435add05/roles/896da438cb834959bd42187c944ea2f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.342448 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93735199d2524322a000a8f09ee05f7b, user_id=303c2c84cd274f25bd61faf0435add05, role_id=896da438cb834959bd42187c944ea2f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:41.345145 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:41.359247 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1465/2923] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:41 2022] PUT /identity/v3/projects/93735199d2524322a000a8f09ee05f7b/users/303c2c84cd274f25bd61faf0435add05/roles/896da438cb834959bd42187c944ea2f7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:41.364398 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb906ff-db66-4d1d-94d2-4010a7bc3a84 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.364760 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb906ff-db66-4d1d-94d2-4010a7bc3a84 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.365008 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb906ff-db66-4d1d-94d2-4010a7bc3a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.384907 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6fb906ff-db66-4d1d-94d2-4010a7bc3a84 None None] MFA Rules not processed for user `7935e75bd19b46fd9203b785ff55ae84`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:41.385644 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6fb906ff-db66-4d1d-94d2-4010a7bc3a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:41.416405 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:41.448759 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1459/2924] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:41.459464 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:41.461281 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:41.466764 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:41.524317 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddee83-e472-41ea-9932-b339e80d108d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:41.528119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:41.528761 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:41.529043 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:41.529320 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:41.539314 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:41.540810 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:41.550314 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1466/2925] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:45.959592 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86965d0-caa9-4f9f-8bd2-c0c18f22e989 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:45.960196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86965d0-caa9-4f9f-8bd2-c0c18f22e989 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:45.960196 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86965d0-caa9-4f9f-8bd2-c0c18f22e989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:45.993692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f86965d0-caa9-4f9f-8bd2-c0c18f22e989 None None] MFA Rules not processed for user `303c2c84cd274f25bd61faf0435add05`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:45.995050 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f86965d0-caa9-4f9f-8bd2-c0c18f22e989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:46.077468 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1460/2926] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:46.081864 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:46.093472 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c455-6c54-43be-a040-659de02f1fbd service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:46.101089 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:46.101714 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:46.101995 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:46.102269 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:46.102910 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:46.105560 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:46.137952 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1467/2927] 10.222.0.26 () {60 vars in 1153 bytes} [Sun Sep 25 01:12:46 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:46.142047 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86965d0-caa9-4f9f-8bd2-c0c18f22e989 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:46.153231 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86965d0-caa9-4f9f-8bd2-c0c18f22e989 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:46.211305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:46.212119 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:46.212427 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:46.212728 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:46.213361 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:46.215277 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:46.239051 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1461/2928] 10.222.0.26 () {60 vars in 1144 bytes} [Sun Sep 25 01:12:46 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 25 01:12:46.244252 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:46.255171 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab60a18-2a80-4b4c-97b5-9ecd8386dbae admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:46.260219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:46.260888 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:46.261172 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:46.261478 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:46.262121 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:46.263739 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:46.274606 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1468/2929] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 25 01:12:46.278845 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:46.291245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac2442-f2d6-4c11-bdf7-ad37149d432f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:46.295534 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:46.296191 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:46.296505 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:46.296781 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] PATH_INFO: `/v3/projects/d18fbeaf9c1f4946b4172b4f2770a0f9/users/6b61d1d142ba408a9dfd08f31b6d1086/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:46.315364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18fbeaf9c1f4946b4172b4f2770a0f9, user_id=6b61d1d142ba408a9dfd08f31b6d1086, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:46.318423 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:46.342177 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1462/2930] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:46 2022] PUT /identity/v3/projects/d18fbeaf9c1f4946b4172b4f2770a0f9/users/6b61d1d142ba408a9dfd08f31b6d1086/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:46.343450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:46.356337 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ec226-6960-415d-93c3-a0259926b428 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:46.417301 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc851b-8b9d-4612-a3dc-1901021c5802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:46.417921 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc851b-8b9d-4612-a3dc-1901021c5802 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:46.418203 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc851b-8b9d-4612-a3dc-1901021c5802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:46.418476 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc851b-8b9d-4612-a3dc-1901021c5802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:46.419113 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc851b-8b9d-4612-a3dc-1901021c5802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:46.420837 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc851b-8b9d-4612-a3dc-1901021c5802 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:46.431558 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1469/2931] 10.222.0.26 () {58 vars in 1122 bytes} [Sun Sep 25 01:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 25 01:12:46.435758 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:46.446221 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213a738-aab1-47e6-99b2-745a52772a0a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:46.450797 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:46.451442 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:46.451723 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:46.451998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] PATH_INFO: `/v3/projects/d18fbeaf9c1f4946b4172b4f2770a0f9/users/6b61d1d142ba408a9dfd08f31b6d1086/roles/3e7f6c18b6fb41099cfa5861f5ae5278` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:46.453924 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18fbeaf9c1f4946b4172b4f2770a0f9, user_id=6b61d1d142ba408a9dfd08f31b6d1086, role_id=3e7f6c18b6fb41099cfa5861f5ae5278)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:46.457286 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:46.467818 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1463/2932] 10.222.0.26 () {60 vars in 1483 bytes} [Sun Sep 25 01:12:46 2022] PUT /identity/v3/projects/d18fbeaf9c1f4946b4172b4f2770a0f9/users/6b61d1d142ba408a9dfd08f31b6d1086/roles/3e7f6c18b6fb41099cfa5861f5ae5278 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:12:51.141838 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876543-4925-4d8e-a83e-98fe43ee27e6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:51.142410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876543-4925-4d8e-a83e-98fe43ee27e6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:51.142410 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876543-4925-4d8e-a83e-98fe43ee27e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:51.173513 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-12876543-4925-4d8e-a83e-98fe43ee27e6 None None] MFA Rules not processed for user `6b61d1d142ba408a9dfd08f31b6d1086`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:51.175200 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-12876543-4925-4d8e-a83e-98fe43ee27e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:51.246902 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:51.259849 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1470/2933] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:12:51.278619 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:12:51.280821 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:51.311503 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:12:51.377137 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901ea2-ac5e-4cbb-9d4a-09d49df4da4c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:12:51.383167 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:12:51.384012 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:51.384575 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:51.385015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:51.455963 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:12:51.457581 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:12:51.466989 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1464/2934] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:12:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:12:52.536875 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:12:52.537705 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:12:52.537980 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:12:52.558831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] MFA Rules not processed for user `303c2c84cd274f25bd61faf0435add05`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 25 01:12:52.559961 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:12:52.567369 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1471/2935] 10.222.0.26 () {58 vars in 958 bytes} [Sun Sep 25 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 25 01:13:01.465586 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=49 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:13:02.552982 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:13:09.303208 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:13:09.362230 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096d203-00b1-4cbd-ae55-95653a429d1e service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:13:09.365853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:13:09.366457 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:13:09.366733 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:13:09.367003 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:13:09.368853 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:13:09.378599 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:13:09.380011 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:13:09.388022 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1465/2936] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:13:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:13:19.384455 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=26 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:13:21.421219 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:13:21.422686 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:13:21.495479 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e45bb-422e-4f1f-b841-6103515b35ab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:13:21.500635 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:13:21.501779 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:13:21.502055 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:13:21.502327 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:13:21.504119 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:13:21.516246 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:13:21.518063 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:13:21.531725 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1472/2937] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:13:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:13:23.159837 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:13:23.170140 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b7b5e-c8e6-48bb-80fe-0d67b82b7573 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:13:23.174461 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:13:23.175085 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:13:23.175374 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:13:23.175645 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:13:23.185884 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:13:23.187345 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:13:23.198015 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1466/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:13:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:13:31.525807 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=32 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:13:33.193576 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:15:50.587181 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:15:50.604601 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b2aa7-1cf3-49b7-9a63-839a0e6fc78e service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:15:50.609231 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:15:50.609849 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:15:50.610134 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:15:50.610412 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:15:50.612248 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:15:50.626325 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:15:50.627781 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:15:50.639451 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1473/2939] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:15:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:15:51.829585 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:15:51.831042 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:15:51.842364 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d294de-dd49-464a-96d7-2726041a04f0 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:15:51.845892 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:15:51.846475 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:15:51.846749 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:15:51.847015 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:15:51.848692 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:15:51.862105 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:15:51.863504 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:15:51.875868 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1467/2940] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:15:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:16:00.634178 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:16:01.869373 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:16:03.946937 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:16:03.955587 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bfe35-d61f-4ef9-8a2d-51a20f49835b service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:16:03.959475 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:16:03.960077 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:16:03.960353 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:16:03.960652 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:16:03.962398 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:16:03.973166 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:16:03.974649 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:16:03.984029 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1474/2941] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:16:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:16:04.070176 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:16:04.079283 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63886f6a-9d7c-41c6-a17e-352b90d32788 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:16:04.083081 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:16:04.083747 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:16:04.083962 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:16:04.084232 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:16:04.086109 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:16:04.100817 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:16:04.102245 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:16:04.113742 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1468/2942] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:16:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:16:13.979342 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:16:14.108863 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:16:28.395702 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:16:28.414985 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dac0a-c833-4531-b328-9aeffe0b71c8 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:16:28.418646 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:16:28.419250 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:16:28.419534 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:16:28.419811 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:16:28.421572 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:16:28.454573 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:16:28.456045 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:16:28.464842 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1475/2943] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:16:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:16:38.460570 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=19 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:17:47.193479 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:17:47.242895 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bed68-fc95-4172-88ab-8885f25dd1cf service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:17:47.248945 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8bf513dc544632a09a2d10988a888f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:17:47.249392 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:17:47.249737 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:17:47.250070 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:17:47.252188 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:17:47.267799 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:17:47.269338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:17:47.281524 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1469/2944] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:17:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:17:51.068964 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:17:51.088697 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb896c-69f3-4e08-b205-6013580c4154 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:17:51.092295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8427b2d4e8c040d0a1b00e25e3b58e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:17:51.092935 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:17:51.093220 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:17:51.093497 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:17:51.167310 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:17:51.169757 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:17:51.192909 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1476/2945] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:17:54.350305 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:17:54.362239 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad1df2-38b2-4208-a6c0-6c27c9b6de3d service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:17:54.367172 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:17:54.367806 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:17:54.368086 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:17:54.368354 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:17:54.370336 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:17:54.382348 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:17:54.383991 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:17:54.396153 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1470/2946] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:17:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 25 01:17:57.664293 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:17:57.675719 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5ab21-4b11-45a0-a318-49c32f84e99e service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:17:57.683049 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:17:57.683734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:17:57.684019 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:17:57.684295 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] PATH_INFO: `/v3/users/303c2c84cd274f25bd61faf0435add05` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:17:57.685481 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303c2c84cd274f25bd61faf0435add05)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:17:57.687403 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:17:57.737819 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303c2c84cd274f25bd61faf0435add05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:57.745340 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303c2c84cd274f25bd61faf0435add05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:57.757930 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303c2c84cd274f25bd61faf0435add05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:57.759094 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303c2c84cd274f25bd61faf0435add05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:57.759690 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303c2c84cd274f25bd61faf0435add05'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:57.771924 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1477/2947] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:17:57 2022] DELETE /identity/v3/users/303c2c84cd274f25bd61faf0435add05 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:17:57.957998 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:17:57.973409 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6806b-faa2-45fa-a872-56fd1f63fd5b service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:17:58.030609 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:17:58.031238 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:17:58.031518 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:17:58.031789 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] PATH_INFO: `/v3/projects/93735199d2524322a000a8f09ee05f7b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:17:58.032925 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93735199d2524322a000a8f09ee05f7b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:17:58.034814 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:17:58.088765 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93735199d2524322a000a8f09ee05f7b', 'request_id': 'req-c38f388d-25aa-4b19-8295-09363fdae3a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.097966 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93735199d2524322a000a8f09ee05f7b', 'request_id': 'req-c38f388d-25aa-4b19-8295-09363fdae3a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.104407 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93735199d2524322a000a8f09ee05f7b', 'request_id': 'req-c38f388d-25aa-4b19-8295-09363fdae3a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.128740 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] The token cache is being invalidate because project 93735199d2524322a000a8f09ee05f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:17:58.128971 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.129583 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.131051 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1471/2948] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:17:57 2022] DELETE /identity/v3/projects/93735199d2524322a000a8f09ee05f7b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:17:58.135831 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:17:58.137284 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:17:58.150029 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753f4b9-24bb-4bd1-939d-337f13f59797 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:17:58.188542 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:17:58.208616 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:17:58.209332 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:17:58.209608 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:17:58.209877 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] PATH_INFO: `/v3/users/6b61d1d142ba408a9dfd08f31b6d1086` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:17:58.211010 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b61d1d142ba408a9dfd08f31b6d1086)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:17:58.213023 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:17:58.259129 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b61d1d142ba408a9dfd08f31b6d1086'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.266591 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b61d1d142ba408a9dfd08f31b6d1086'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.278452 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b61d1d142ba408a9dfd08f31b6d1086'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.279079 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b61d1d142ba408a9dfd08f31b6d1086'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.279683 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b61d1d142ba408a9dfd08f31b6d1086'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.292428 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1478/2949] 10.222.0.26 () {58 vars in 1224 bytes} [Sun Sep 25 01:17:58 2022] DELETE /identity/v3/users/6b61d1d142ba408a9dfd08f31b6d1086 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:17:58.480410 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:17:58.481940 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:17:58.499338 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f388d-25aa-4b19-8295-09363fdae3a7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:17:58.518157 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:17:58.560786 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7935e75bd19b46fd9203b785ff55ae84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3708ba2402cc4ee7abd81255a285d2b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:17:58.561433 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:17:58.561721 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:17:58.561984 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] PATH_INFO: `/v3/projects/d18fbeaf9c1f4946b4172b4f2770a0f9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:17:58.563429 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18fbeaf9c1f4946b4172b4f2770a0f9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:17:58.565163 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:17:58.615804 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18fbeaf9c1f4946b4172b4f2770a0f9', 'request_id': 'req-9a10a2c6-a79d-443f-952c-b071ebaa98f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.624039 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18fbeaf9c1f4946b4172b4f2770a0f9', 'request_id': 'req-9a10a2c6-a79d-443f-952c-b071ebaa98f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.629833 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18fbeaf9c1f4946b4172b4f2770a0f9', 'request_id': 'req-9a10a2c6-a79d-443f-952c-b071ebaa98f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.641190 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] The token cache is being invalidate because project d18fbeaf9c1f4946b4172b4f2770a0f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 25 01:17:58.641471 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.642125 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a10a2c6-a79d-443f-952c-b071ebaa98f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 25 01:17:58.643691 dv-cinder-master-3-smb devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1472/2950] 10.222.0.26 () {58 vars in 1233 bytes} [Sun Sep 25 01:17:58 2022] DELETE /identity/v3/projects/d18fbeaf9c1f4946b4172b4f2770a0f9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 25 01:18:08.288499 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:18:08.638094 dv-cinder-master-3-smb devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:18:24.744734 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 25 01:18:24.746146 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:18:24.835952 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4f56-6f83-48d5-89b0-62f29babbb5a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 25 01:18:24.894576 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6f21d-a8ed-46e7-906a-56a7aaf2d3f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45df63e9f2449be8d13f58100775eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77f655eb34fc4559a31900ece5656154', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 25 01:18:24.895188 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6f21d-a8ed-46e7-906a-56a7aaf2d3f1 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 25 01:18:24.895496 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6f21d-a8ed-46e7-906a-56a7aaf2d3f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 25 01:18:24.895771 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6f21d-a8ed-46e7-906a-56a7aaf2d3f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 25 01:18:24.897553 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ead6f21d-a8ed-46e7-906a-56a7aaf2d3f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 25 01:18:24.908848 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6f21d-a8ed-46e7-906a-56a7aaf2d3f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 25 01:18:24.910450 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6f21d-a8ed-46e7-906a-56a7aaf2d3f1 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 25 01:18:24.921116 dv-cinder-master-3-smb devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1479/2951] 10.222.0.26 () {60 vars in 1365 bytes} [Sun Sep 25 01:18:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 25 01:18:34.916867 dv-cinder-master-3-smb devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=67 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}